DOCUMENT  RESUME 

ED  347  970  1R  015  706 


AUTHOR 
TITLE 


INSTITUTION 

PUB  DATE 
NOTE 


PUB  TYPE 


Simonson,  Michael  R.,  Ed.;  Jurasek,  Karen  A.,  Ed. 
Proceedings  of  Selected  Research  and  Development 
Presentations  at  tile  Convention  of  the  Association 
for  Educational  Communications  and  Technology  and 
Sponsored  by  the  Research  and  Theory  Division  (14th, 
Washington,  D.C*,  February  5-9,  1992). 
Iowa  State  Univ*  of  Science  and  Technology r  Ames. 
Coll.  of  Education. 
Feb  92 

970p.;  For  the  individual  papers,  see  ED  335  000,  ED 
343  570,  ED  345  697,  IR  015  707-710,  and  IR  015 
712-778.  For  the  1991  proceedings,  see  ED  334  969. 
Collected  Works  -  Conference  Proceedings  (021)  — 
Viewpoints  (Opinion/Position  Papers,  Essays,  etc.) 
(120)  —  Reports  -  Research/Technical  (143) 


EDRS  PRICE 
DESCRIPTORS 


IDENTIFIERS 


MF07/PC39  Plus  Postage. 

Adoption  (Ideas)?  Attitudes;  Cognitive  Processes; 
^Computer  Assisted  Instruction;  Computer  Simulation; 
Critical  Theory;  Distance  Education;  Elementary 
Secondary  Education;  Ethics;  ^Feedback;  Higher 
Education;  High  Risk  Students;  ^Hypermedia; 
*  Instructional  Design;  Interactive  Video;  Intermode 
Differences;  Learning  Strategies;  *Media  Research; 
Metacognition;  Research  Methodology 
Channel  One;  Learning  Environments 


ABSTRACT 

The  papers  in  this  collection  of  71  individual  papers 
and  3  symposiums  represent  some  of  the  most  current  thinking  in 
educational  communications  and  technology.  For  the  first  tim*,  a 
selected  number  of  development  papers  sponsored  by  the  Division  for 
Instructional  Development  are  included.  Topics  discussed  include  the 
effects  of  various  conditions  on  the  learner  and  learning,  including 
aptitude  treatment  interactions;  learning  strategies;  teacher 
planning;  instructional  design  and  development;  teaching  strategies; 
the  teacher1 s  role;  teacher  education;  research  methodology; 
networking;  computer-assisted  instruction;  hypermedia;  distance 
education;  teleconferencing;  interactive  video;  computer  simulations; 
and  the  instructional  environment.  The  three  symposiums  focused  on 
ethics,  social  considerations  in  educational  computing,  and  the 
significance  of  the  Channel  One  experiment.  While  research  reports 
predominate,  some  reviews  of  the  literature  and  opinion  papers  are 
included.  This  volume  also  includes  a  list  of  Research  and  Theory 
Division  officers;  a  list  of  the  reviewers  of  the  papers  for  the 
conference?  a  list  of  proceedings  from  the  year  1979  together  with 
their  ERIC  document  numbers;  a  description  of  the  Association  for 
Educational  Communications  and  Technology  (AECT)  and  a  membership 
enrollment  form;  a  table  of  contents;  and  author  and  descriptor 
indexes.  (BBM) 


US  DEPARTMENT  OF  EDUCATION 

t  in  .    l>(  [  (ftiriiiior.al  HHwarch  and  tmprovp/nrnl 

fcOUCAfuNAl  HKSOUHCfcS  INFORMATION 

i:!;NTfP{f:RtC) 

l'i>'.  ,K»i  mnenl  has  »>*»**n  reproduced  its. 
/i'<  t.'Mt'd  !ri>m  (he  pwson  of  organisation 

,  Minor  chantjus  i\n<v  butth  made  to  improve 
reproiludion  (JoQIity 

•  JWlsul  fit**  c jf  o|)inmfisSlHUHi»n|hib  Jm.« 
mi»n'  do  no!  n^r »»v.,o ily  r»prHSt*nl  ofinat 
Of  Ri  pos.hon  or  lX)ln  y 


14th  ANNUAL 
PROCEEDINGS 
of 

SEI  JfiCTED  RESEARCH  AND  DEVELOPMENT  PRESENTATIONS 

at  the  1992  Convention  of  the 
Association  for  Educational  Communications  and  Technology 
Sponsored  by  the  Research  and  Theory  Division 

in 

Washington,  D,C, 


Edited  by: 

Michael  R,  Simonson 
Professor  of  Secondary  Education 


and 


Karen  A,  Jurasek 
Teaching  Assistant 


Iowa  State  University 
College  of  Education 
Instructional  Resources  Center 
Lagomarcino  Hall 
Ames,  Iowa  50011 

(515)  294-6840 


"PERMISSION  TO  REPRODUCE  THIS 
MATERIAL  HAS  BEEN  GRANTED  BY 


Michael  Sltno n a o n 


TO  rHE  EDUCATIONAL  RESOURCES 
INFORMATION  CENTER  (ERIC)  " 


BEST  COPY  AVAILABLE 


PREFACE 


For  the  fourteenth  year,  the  Research  and  Theory  Division  of  the  Association  for 
Educational  Communications  and  Technology  (AECT)  is  sponsoring  the  publications  of  these 
Proceedings.  Papers  published  in  this  volume  were  presented  at  the  national  AECT  Convention  in 
Washington,  D.C.  A  limited  quantity  of  this  volume  were  printed  and  sold.  It  is  also  available  on 
microfiche  through  the  Educational  Resources  Information  Clearinghouse  (ERIC)  system, 

REFEREEING  PROCESS:  All  research  papers  selected  for  presentation  at  the  AECT 
convention  and  included  in  this  Proceedings  were  subjected  to  a  rigorous  blind  reviewing  process, 
Proposals  were  submitted  to  Steven  Ross  and  Gary  Morrison  of  Memphis  State  University  or  Gary 
Anglin  of  the  University  of  Kentucky.  All  references  to  author  were  removed  from  proposals 
before  they  were  submitted  to  referees  for  review.  Approximately  fifty  percent  of  the  manuscripts 
submitted  for  consideration  were  selected  for  presentation  at  the  Convention  and  for  Publication  in 
these  Proceedings.  The  papers  contained  in  this  document  represent  some  of  the  most  current 
thinking  in  educational  communications  and  technology, 

For  the  first  time  a  selected  number  of  development  papers  sponsored  by  the  Division  for 
Instructional  Development  (DID)  are  included  in  this  Proceedings.  The  most  important 
instructional  development  presentations  were  selected  by  the  DID  program  chairs  for  publication. 

This  volume  contains  two  indexes.  The  first  is  an  author  index;  the  second  is  a  descriptor 
index.  The  index  for  volumes  1-6  (1979-84)  is  included  in  the  1986  Proceedings,  and  the  index  for 
volumes  7-10  is  in  the  1988  Proceedings. 

M.  R.  Simonson 
Editor 
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Association  for  Educational 
Communication  and  Technology 

WhatlsAECT? 

AECT  is  the  only  national,  professional  associa- 
tion dedicated  to  the  improvement  of  instruction 
through  the  effective  use  of  media  and  technol- 
ogy. AECT  assists  its  members  in  using  technol- 
ogy in  their  jobs  and  to  enhance  the  learning 
process. 

Who  belongs  to  AECT? 

□  Media  Specialists 

□  Educators 

□  Instructional  designers 

□  Learning  resource  specialists 

□  Curriculum  developers 

□  Television  producers  and  directors 

□  Communications  specialists 

□  Education  administrators 

□  Others  who  require  expertise  in 
instructional  technology 

AECT  History 

AECT  began  as  the  Department  of  Audio-Visual 
Instruction  at  the  National  Education  Association 
in  1923,  in  the  days  when  visual  aids  were  black- 
boards and  easels.  In  1947,  as  educators  were 
adapting  technology  used  to  train  World  War  II 
service  personnel  forthe  classroom,  the  name  of 
the  organization  became  the  Department  of  Vis- 
ual Instruction  (DAVI).  Twelve  years  later,  DAVI 
became  an  affiliate  of  the  NEA  and  finally  the 
autonomous  association,  AECT,  in  1974. 


Today,  AECT  keeps  an  eye  on  the  future  of  instruc- 
tional technology  while  assisting  educators  with  the 
changes  and  challenges  that  face  them  now.  AECT 
members  are  professionals  devoted  to  quality  edu- 
cation. They  care  about  doing  their  jobs  better  and 
want  to  ambrace  new  methods,  new  equipment,  and 
new  techniques  the.*  assist  learning. 

AECT  Affiliates 

AECT  has  45  state  and  1 1  national  affiliates,  and  has 
recently  established  several  chapters  surrounding 
major  universities  and  metropolitan  areas.  These 
affiliated  organizations  add  a  localized  dimension  to 
your  AECT  membership  and  allow  for  more  interac- 
tion among  your  colleagues.  For  more  details  on 
chapters  and  affiliates  in  your  area,  contact  the 
AECT  National  Office. 

AECT  National  Convention  and 
INFOCOMM  international  Exposition 

Each  year,  AECT  brings  top  speakers  to  exciting 
locations,  and  presenis  over 300  sessions  and  special 
events  to  provide  the  best  training  available  in  the 
use  of  media  in  education  and  instruction.  The 
convention  features  the  INFOCOMM  International 
Exposition  ,  an  extravaganza  of  over  300  compa- 
nies showing  the  latest  products  to  use  in  education 
and  training. 

Research  and  Theory  Division  (RTD)  improves 
the  design,  execution,  utilization,  evaluation,  and 
dissemination  of  audiovisual  communications  re- 
search; advises  educators  on  the  effective  utilization 
of  the  results  of  the  research;  and  promotes  applied 
and  theoretical  research  on  the  use  of  media. 
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ID  Knowledge  Structure,  Lesson  Planning  and  Teacher  Performance 

Considerable  Interest  has  been  expressed  recently  among  professors  of 
instructional  design  regarding  the  potential  contributions  of  their  Held  for 
practicing  teachers.  Calls  for  including  ID  skills  in  programs  of  teacher  training 
can  be  found  in  various  writings  (Bielby  1974;  Stolovitch,  1980;  Earle,  1985;  Dick  & 
Carey,  1985;  Dick  and  Reiser,  1989;  and  Klein,  1991).  Several  undergraduate 
programs  now  include  more  substantive  treatments  of  instructional  design  (albeit, 
adapted  for  teachers),  rather  than  the  very  incomplete  treatment  that  typically 
occurs  in  preservice  teachers'  obligatory  course  in  Educational  Psychology. 

While  recommendations  for  teaching  teachers  a  systems  approach  to 
instruction,  which  would  include  associated  ID  concepts,  processes  and  skills,  is 
certainly  logically  defensible,  research  to  support  the  efficacy  of  such  an  approach 
in  the  typical  planning  routines  and  subsequent  instruction  of  teachers  is  lacking. 
The  evidence  that  does  exist  is  based  on  the  literature  of  teacher  thinking  and 
planning  (Yinger,  1979;  McCutcheon,  1980;  Clark  &  Peterson,  1986;  Brown,  1988), 
and  it  consistently  indicates  that  most  teachers  engage  in  a  planning  process  that 
is  incongruent  with  significant  aspects  of  generic  1SD  models.  Teachers  do  not 
typically  report  the  use  of  the  linear  rational  planning  model  that  characterizes 
models  Tor  the  systematic  design  of  instruction.  Such  findings  present  a  challenge 
to  the  lipid  of  instructional  systems  design  and  to  those  who  wish  to  promote 
beneficial  applications  of  institutional  design  principles  to  preservice  and 

inservice  teachers  . 

In  the  teacher  training  program  at  the  Uni/ersity  of  North  Carolina  at 
Wilmington,  all  students  must  take  a  course  in  instructional  design,  and  a  course 
in  evaluation,  and  earn  a  grade  of  C  or  better  in  each.  The  undergraduate  ID 
course  teaches  a  systems  approach  to  instruction  using  an  adaptation  of  Dick  and 
Carey's  ID  model,  and  emphasizing  Gagne's  taxonomy  and  events  of  instruction. 
Aithough  all  students  complete  the  two-course  sequence,  some  variability  can  be 
expected  among  students'  cognitive  and  attitudinal  learning  o   comes,  and  in  the 
ways  that  these  learnings  are  nn?anized  and  internalized.  Satisfactory 
performance  on  the  discrete  intellectual  skills  and  verbal  information  components 
of  courses  may  belie  possibly  serious  deficiencies  in  mastery  of  the  higher  level 
cognitive  structure  of  the  course  or  topic  on  the  part  of  some  students.  Such 
deficiencies  may  reflect  misconceptions  within  the  knowledge  structure  formed  by 
students,  even  among  those  earning  the  same  course  grade. 

Thus,  a  course  exit  task  which  allows  students  to  represent  their  own 
cognitive  structure  for  ID  concepts,  principles,  and  relationships  may  better  reveal 
students'  degree  of  acquisition  of  the  deep  structure  of  the  discipline  than  end-of- 
course  grades  (Novak  &  Gowin,  1984;  Jonassen,  1987;  Wallace  &  Minlzes,  1990). 
Fortunately  there  are  a  variety  of  mapping  strategies  that  have  been  described  and 
evaluated  (Jonassen,  Belssner,  Kenny,  Jost,  Reid,  &  Yaccl,  1990;  Yacci  1990). 

Gaining  insight  into  preservice  students'  knowledge  structures  for  ISD 
could  improve  our  understanding  of  the  ways  in  which  they  subsequently  utilize 
this  knowledge  for  teacher  planning.    If  there  are  qualitative  differences  in 
students'  conceptions  of  an  ISD  approach  to  instruction,  they  might  be  related  to 
overall  teacher  performance. 

The  following  study  was  conducted  to  address  these  questions.    First,  can 
preservice  teachers'  knowledge  structure  of  ISD  concepts,  principles  and  attitudes 
be  evaluated?  Second,  are  preservice  teachers  who  have  an  accurate  conceptual- 
ization of  instructional  design  (knowledge  structure)  more  likely  than  those  with 
weak  knowledge  structures  to  adopt  a  process  for  planning  instruction  that  is  more 
consistent  with  ISD  models?  Third,  are  teachers  who  implement  more  elements  of 
ID  in  their  Instructional  planning  judged  to  be  more  effective  teachers?  Fourth, 
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what  do  student  teachers  report  about  their  beliefs  regarding  the  efficacy  of  ISD 
principles  for  planning  and  delivering  instruction  and  how  they  actually  apply 
them? 

METHOD       Subjects  were  preservice  teacher  interns  at  the  University  of  North 
Carolina  at  Wilmington.  All  had  completed  a  course  in  instruction  i  design  and 
a  course  in  evaluation  with  a  grade  of  C  or  better.  In  the  research  reported  here, 
Novak  and  Gowin's  (1984)  cognitive  mapping  task  has  been  partially  modified 
(see  Strahan's  semantic  ordered  trees,  1989).  A  recent  study  by  Beyerback  and 
Smith  (1990)  demonstrated  the  feasibility  of  using  cognitive  mapping  tasks  to 
measure  changes  in  preservice  teachers1  conceptions  of  broad  topics  (effective 
teaching).  Subjects  were  given  a  set  of  27  starter  words  relating  to  teacher 
planning,  and  a  set  of  14  linking  verbs/  phrases  and  were  asked  to  create  a 
cognitive  map. 

STARTER  WORDS:  attitude,  classroom  management,  events  of  instruction, 
formative  evaluation,  individual  differences,  individualized  instruction, 
instructional  program,  instructional  strategies,  intellectual  skills,  large  group, 
learning  outcomes,  lesson  plans,  mastery  learning,  mental  plans,  media, 
motivation,  objectives,  remediation,  revision,  routines,  subskills,  tests,  textbooks, 
transfer  of  learning,  transitions,  unit  plans,  verbal  information. 
LINKING  VERBS/PHRASES:  is  subordinate  to,  is  example  of,  is  similar  to,  is 
needed  for,  is  based  on,  takes  place  when,  helps,  comes  before,  occurs 
simultaneously  with,  involves,  leads  to,  facilitates,  affects,  indicates  that. 

After  reviewing  examples  of  3  cognitive  maps  and  practicing  by  creating  two 
simple  cognitive  maps  (of  dogs  and  classroom),  the  interns  were  asked  to  prepare  a 
cognitive  map  for  Teacher  Planning.    Subjects  wen?  instructed  to  use  as  many  or 
as  few  of  the  starter  words  and  linking  phrases  as  they  desired  and  to  feel  free  to 
add  whatever  concepts  they  deemed  important  to  express  their  conception  of  teacher 
planning. 

Six  students  completed  a  cognitive  mapping  task  on  teacher  planning 
during  the  summer  prior  to  student  teaching.    After  student  teaching  they 
constructed  a  second  cognitive  map  of  teacher  planning  and  completed  a 
questionnaire  on  teacher  planning.  Six  additional  fall  semester  teacher  interns 
also  completed  the  questionnaire.  Student  exit  performance  was  determined  by 
grades  in  student  teaching  and  by  university  supervisors1  summative  evaluation 
checklists. 

Qualitative  evaluations  were  used  to  judge  the  sophistication  of  knowledge 
structure  for  teacher  planning  represented  in  the  cognitive  maps.  The  following 
criteria  were  used:  a)  number  of  concepts  used*  b)  number  and  coherence  of  items 
per  cluster,  c)  validity  of  relationships  specified,  d)  comprehensiveness  or  number 
and  significance  of  concepts  included/omitted,  and  e)  the  narrative  statements  that 
accompanied  each  map. 

RESULTS      There  was  considerable  variability  in  the  sophistication  and 
stability  of  the  cognitive  maps,  as  well  as  in  the  degree  of  change  from  pre-  to  post- 
administration  of  the  mapping  task.  Three  students1  pre-and  post-maps  have  been 
selected  for  discussion  (  see  Figures  1,  2  and  3y.  Maps  1A  and  IB  reveal  a  good 
understanding  of  the  fundamental  precepts  of  instructional  systems  design.  They 
exhibit  good  detail  and  are  well  organized.  Map  1A  addresses  the  relationships 
among  the  major  aspects  of  planning  and  emphasizes  individual  differences.  In 
map  IB  one  sees  a  more  streamlined  rendition  of  teacher  planning.  Classroom 
management  is  now  portrayed  as  one  of  three  critical  dimensions  in  planning, 
along  with  the  dimension  of  individualized  instruction.  This  map  does  portray  a 
systematic  planning  approach  and  conveys  a  heightened  awareness  and  urgency 
regarding  the  individual  learner  and  the  environmental  context  of  learning  in 
schools.  Concern  for  the  classroom  environment  is  a  salient  feature,  as  expressed 
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by  the  statement,  "...  should  be  comfortable;  children  should  be  able  to  ask 
questions  and  take  risks.  The  class  should  be  success  oriented." 

In  map  2A  one  finds  that  a  large  number  of  planning  concepts  have  been 
included,  but  the  relationships  among  these  concepts  are  not  clearly  made.  The 
main  organizing  frame  for  this  map  b  events  of  instruction,  but  it  is  not  entirely 
clear  how  this  concept  is  being  used  In  short,  the  process  of  plai  ning  is  not 
revealed,  nor  is  there  any  indication  that  an  instructional  systems  design  model  is 
operating  to  coordinate  thinking  about  planning.  In  the  post-  student  teaching  map 
(2B).  the  dominant  organizer  of  planning  concepts  nas  shifted  to  mental  plans. 
Once  again  terms  are  simply  classified  into  groups  that  share  some  reasonably 
common  theme.  The  poor  articulation  in  the  supplemental  narratives  for  these 
maps  is  consistent  with  the  conclusion  that  maps  2A  and  2B  exhibit  a  much  mere 
limited  view  of  teacher  planning  and  give  little  evidence  of  an  understanding  of 
instructional  systems  design. 

2A:  Teacher  planning  consists  of  many  components.  It  is  important 
that  the  instructional  program,  instructional  strategies,  lesson 
plans,  mental  plans  and  individual  instruction  all  be  a  part  of  the 
events  of  instruction.  The  instructional  program  should  be 
integrated.  If  it  is  integrated  the  learning  outcomes  will  flow... 
2B:  Start  olT  with  mental  plans  -  big  areas  of  starting  off:  events  of 
instruction,    instructional  program,  lesson  plans,  management, 
revisions  and  evaluation.  Then  I  have  eacn  subdivision  broken 
down  with  what  should  be  included. 

In  Maps  3A  and  3B  we  see  a  highly  systematic  process  of  planning  and 
abundant  evidence  of  the  application  of  the  essential  elements  of  ISO.  The 
importance  of  making  revisions  is  a  prominent  feature  of  map  3A,  as  is  an 
apparently  greater  emphasis  on  planning  for  classroom  management  within  the 
overall  context  of  teacher  planning.    Map  3B  has  been  trarotfonned  into  more  of  a 
flowchart  for  planning  beginning  with  global  or  year-long  plans.  The  revision 
process  continues  to  be  vipwed  as  critical  to  planning  as  does  the  essential 
contributions  of  classroom  management  and  learner  characteristics  (including 
attitudes  and  motivation)  to  planning  decisions  (instructional  strategies,  selection 
of  materials  and  media,  and  application  of  events  of  instruction).  This  student's 
narrative  reveals  the  prominence  of  1SD  principles  in  her  approach  to  planning  for 
instruction  as  well  as  the  tentative  nature  of  planning  for  a  class  of  third  graders: 
Teacher  planning  is  a  complex  part  of  this  profession.  It  is  also  not 
something  that  can  be  done  once  and  that's  it.  Planning  is  a  process 
which  often  requires  revision.  New  thoughts  are  generated  daily  by 
what  you've  seen  your  students  do.  1  found  myself  constantly 
questioning  my  objectives,  strategies  and  evaluation  measures  t' 
see  if  1  was  providing  for  all  children's  success  in  the  classroom. 
After  reviewing  the  state  curriculum  guide  and  establishing  year 
long  goals,  1  was  then  able  to  generate  a  six  week  plan  of 
instruction.  My  daily  plans  were  then  derived  from  the  six  weeks 
plan.  1  selected  my  objectives  and  asked  myself  questions  about 
expected  learning  outcomes.  Then  keeping  my  students'  learning 
styles  in  mind,  1  designed  instructional  strategies  for  providing  a 
learning  rich  lesson.  Most  of  my  plans  included  cooperative 
learning  activities.  Following  instruction  1  used  the  test  initially 
designed  in  the  planning  process.  If  1  altered  or  revised  during  any 
stage,  my  tests  also  were  revised.  I  constantly  tried  to  transfer  the 
children's  knowledge  and  searched  for  ways  to  make  this 
knowledge  most  meaningful  in  their  lives. 
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Each  student  was  supervised  and  evaluated  by  one  of  three  university 
faculty  members.  Five  of  the  student  teachers  received  a  grade  of  'A'  in  the  student 
teaching  course;  the  other  student  received  a  'B'.  The  student  teacher  whose 
cognitive  maps  (2A  and  2B)  were  judged  to  be  least  sophisticated  received  a  grade  of 
'B'.   All  students  were  rated  excellent  in  the  function  of  Planning  for  Instruction. 
Since  there  was  minimal  variability  In  the  ratings  and  grades  of  the  six  teachers, 
no  relationships  could  be  established  between  these  outcome  measures  and 
diffprenres  detected  in  the  sophistication  of  cognitive  maps. 
QUESTIONNAIRE  RESULTS         There  were  differences  between  the  patterns 
of  responses  of  the  6  interns  who  taught  In  either  a  kindergarten  or  first  grade  class 
(kdg-3;  1st-  3).  and  the  6  interns  who  taught  in  the  third  grad*  (3),  fourth  grade  (2)  or 
fifth  grade  (1).  v^nsequently,  the  average  ratings  (1-5)  for  each  of  the  18 
questionnaire  iU...:  are  summarized  separately  for  these  two  groups.  See  Table  1. 

Teachers  in  K-l  classes  expressed  strongest  agreement  with  items:  8 
(...mindful  to  take  account  of  prerequisites  in  planning  and  sequencing 
instruction);  11  (There  is  an  essential  relationship  among  objectives,  instruction 
aid  evaluation);  12  (...  much  of  teacher  planning  is  never  put  on  paper);  13  (...  one 
can  think  in  terms  of  types  of  learning  outcomes  without  actually  writing  down 
behavioral  objectives);  and  16  (...  planning  begins  with  a  mental  conception  of  an 
objective,  not  a  behavioral  objective).  These  primary  grade  teachers  were  very 
emphatic  about  the  idea  of  planning  without  putting  their  thought  s  in  writing  (4.67). 
They  expressed  slight  disagreement  with  the  following  items:  1  and  2  (...  my  study 
of  Gagne's  events  of  instruction  has  helped  me  to  plan  effective  instructional 
activities;  deliver  effective  instruction);  4  (...  my  study  of  Gagne's  taxonomy  of 
learning  has  helped  me  to  deliver  effective  instruction);  9  (I  am  careful  In 
planning  lessons  to  include  the  appropriate  events  of  instruction  that  apply  to  the 
domain  of  learning  being  taught);  10  (...I  incorporate  the  external  conditions  of 
learning  that  are  relevant  to  the  domain  of  objectives  in  the  lesson);  and  14  (What  1 
write  in  my  lesson  plans  is  almost  always  translated  into  what  actually  occurs 
when  I  teach).  It  Is  also  noteworthy  that  the  K-l  teachers  Indicated  slight  agreement 
with  item  18  ( I  really  have  not  found  the  instructional  design  concepts  and  skills  to 
be  that  relevant  to  teaching). 

While  the  perceptions  of  both  groups  were  quite  similar  for  many  items, 
disparities  were  observed  for  other  items.  Teachers  of  grades  3,  4  or  5  also 
indicated  strongest  agreement  with  items  8, 11, 13,  and  16  (see  above).  However,  a 
discrepancy  was  observed  between  the  responses  of  the  two  groups  of  teachers  for 
items  1, 2,  4.  9, 10, 12,  and  18.  Overall  there  was  a  tendency  for  grades  3-5  teachers 
to  report  more  application  of  ISD  principles,  to  view  more  positively  the 
contributions  of  their  coursework  in  instructional  design  to  their  planning  and 
teaching,  and  to  be  less  adamant  in  the  view  that  much  of  teacher  planning  is 
never  put  on  paper. 

Additional  responses  to  open  ended  questions  revealed  that  the  preservice 
teachers  used  a  comprehensive  approach  to  the  planning  function  of  teaching.  In 
one  intern's  words: 

Teacher  planning  is  a  very  complex  process.  As  a  teacher  you  have 
to  know  your  students'  individual  strengths  and  needs.  You  have  to 
develop  strategies  to  teach  concepts  according  to  those  needs 
(positive  planning);  evaluate  and  re-evaluate  strategies;  study 
knowledge  you  want  to  teach,  know  prior  knowledge  that  is  needed 
in  order  to  learn  new  info.  What  you  want  your  children  to  learn 
should  be  noted  (process  and  product  you  would  like  to  see)  and  then 
evaluate  to  see  if  t\\is  happened... 
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A  potentially  important  insight  into  another  facet  of  certain  teachers' 
planning  is  conveyed  by  this  response  to  the  question,  "Did  the  way  in  which  you 
used  objectives  in  your  general  planning  and  specifically  in  your  written  le«*on 
plans  change  over  the  course  of  your  student  teaching  semester?: 

When  integrating  our  subjects  we  would  first  pick  out  the  concept 
we  felt  to  be  Important  and  those  we  wanted  to  teach.  Through  those 
concepts  we  developed  our  goals,  objectives,  activities,  etc.  For 
example  the  concept,  'conflict1:  Social  Studies  -  Revolutionary 
War;  Math  •  Math  Sandwich  Shops;  Health  :  Independent  Me; 
Literature  -Mrs,  Frisby  and  the  Rats  of  Ninth. 
The  thematic  approach  to  teaching  described  here  seems  to  place  more  emphasis  on 
the  goal  or  achieving  interdisciplinary  learnings.  It  suggests  a  more  holistic  ,  less 
discrete  view  of  the  outcomes  of  instruction.  The  whole  language  approach  that  is 
currently  widely  advocated  for  teaching  the  language  arts  curriculum  is  yet 
another  example  of  a  general  teaching  methodology  embedded  in  a  philosophy  of 
teaching  and  learning  that  emphasizs  the  integrated,  holistic,  constructivist  nature 
of  learning. 

The  essential  nature  of  mental  planning  either  In  tandem  with  or  in  lieu  of 
more  formal  written  lesson  planning  was  repeatedly  underscored.  Here  are  three 
responses  to  the  questions:  "Do  you  engage  in  mental  planning?  Describe  how  you 
do  this  type  of  planning. 

Mental  planning  takes  up  to  75%  of  your  time  if  you  are  interested  or 
concerned  about  how  you  teach.  A  continual  recording  in  your 
mind  runs  about  how  students  will  react,  what  you  need  to 
remember,  etc. 
Another  teacher  expressed  it  this  way: 

I  would  say  that  1  am  always  mentally  planning  but  1  am  a  person 
who  has  to  put  things  on  paper  in  order  fir  them  to  make  sense.  ...1 
was  constantly  revising  and  editing  my  plans.  It  is  a  constantly 
changing  situation  that  almost  forces  one  to  plan  with  every  free 
moment's  thoughts. 

And  tills  teacher's  comment  underscores  the  importance  of  mental  planning  and 
the  urgency  (or  perhaps  ,  impatience  with  planning)  that  some  teachers  bring  to 
their  work: 

Yes,  most  definitely  -  I  can  explain  things  verbally  much  better 
than  trying  to  write  down  what  all  I'm  thinking  about.  1  think  about 
an  objective,  or  task  which  must  be  performed,  learned.  Form 
different  activities  that  would  enhance  this  learning.  Write  them 
down.  Form  a  type  of  test  idea  that  would  show  how  much  learning 
has  occurred.  Then  make  notes  -  Do  itl 
DISCUSSION  What  emerges  from  these  data  is  a  picture  of  teachers  who 

are  planful,  organized  and  very  concerned,  as  one  would  expect,  with  the  most 
salient  aspects  of  teaching:  a)  th*  learners,  and  b)  the  teaching  activities  -  with 
getting  the  work  done.  While  there  were  several  comments  made  about  the 
inefficiency  and  questionable  utility  of  writing  detailed  objectives  or  detailed 
lesson  plans,  these  preservice  teaciiers  made  marjy  references  to  their  use  of 
objectives  in  planning.  They  are  veiy  cognizant  of  the  complexities  of  their  work 
and  oi  the  necessity  of  taking  into  account  a  number  vf  critical  variables  when 
planning.  Although  they  may  not  consistently  plan  according  to  a  linear  ID 
model,  most  reported  tliat  they  systematically  considered  crucial  planning 
variables  and  in  general  described  their  mental  planning  in  terms  of  a  systems 
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view  of  teaching,  for  example,  consider  this  intern's  statement: 

I  feel  strongly  that  a  teacher  has  to  plan  very  diligently  for  the 
students  to  be  successful.  However,  I  don't  feel  that  an  objective  has 
to  be  written  in  a  formal  form  for  learning  to  take  place  Maybe  I  do 
things  backwards  -  I  always  think  about  my  students1  learning  and 
where  1  want  them  to  go  and  what  I  want  them  to  get  out  of  it,  but  (as  I 
sit  back)  I  think  I  just  do  it  in  a  different  order. 
Thiagarayan  (1976)  makes  a  similar  point  in  aqjiring  for  the  acceptability  of  a 
more  flexible  application  of  the  systems  approach  to  the  design  of  instruction.  This 
seems  particularly  appropriate  for  the  fluid  world  of  the  classroom  teacher. 

Several  other  findings  are  noteworthy.  At  least  during  the  highly  self- 
conscious  period  of  their  student  teaching  experience,  these  interns  incorporate 
some  aspects  of  formative  evaluatioa  This  is  evidenced  by  their  propensity  for 
regular  reflection  on  their  instructional  successes  and  mishaps;  and  a  readiness, 
if  not  a  definite  expectation,  for  revising  their  instructional  activities.  Also, 
written  objectives  give  way  to  time  constraints  and  to  the  primacy  of  mental 
planning,  with  brief  notes  being  used  as  cues  for  teaching.  Lastly,  it  is  quite 
apparent  that  the  preservice  teachers  in  this  small  sample  consistently  placed  a 
high  premium  on  attending  carefully  to  their  learners  as  they  engaged  in  the 
process  of  planning.  As  one  student  remarked,  "I  determine  the  needs  of  the 
student  and  seek  to  engage  in  learning  activities  that  will  effectively  meet  those 
needs.  " 

To  better  understand  these  preservice  elementary  teachers'  overriding 
concern  for  learner  characteristics,  it  is  relevant  to  recall  Walter  Dick's 
observation  (1981)  in  an  article  about  future  trends  and  issues  in  instructional 
design: 

Most  instructional  design  models  are  intended  to  have  broad 
application.  Therefore,  they  are  not  specific  to  ai\y  content  or  to  any 
particular  set  of  learners  or  instructors.  As  such,  instructional 
design  models  (and  instructional  designers)  sometimes  give  the 
appearance  of  ignoring  the  role  of  the  student  and  the  teacher  in  the 
learning  process.  It  may  be  hypothesized  that  in  the  decade  ahead 
there  will  be  more  emphasis  on  the  people  who  implement  the 
learning  system.  Emphasis  will  be  manifested  in  terms  of  a 
greater  knowledge  of  the  general  characteristics  of  the  learners  who 
are  being  served  -  their  motivations,  their  learning  habits,  and 
their  preferences.  Designers  will  avoid  the  stereotypes  of  learners 
and  work  directly  with  them,  (p.32) 
This  prediction  for  instructional  designers  may  in  fact  characterize  the  everyday 
reality  of  elementary  school  teachers.  Certainly  the  image  that  emerges  of 
preservice  elementary  education  teadiers  from  their  cognitive  maps  and  from  self- 
reports  of  their  mental  planning  behavior  constitutes  a  strong  affirmation  of 
Dick's  earlier  insight. 

One  certainly  gains  a  renewed  appreciation  of  the  complex  and  cognitively 
demanding  nature  of  good  teaching.  Teacher  planning  is  without  question  a  broad 
and  complex  topic  that  requires  the  integration  of  a  number  of  elements  including 
lesson  planning.   In  conceptualizing  planning,  teachers  must  consider  how  they 
integrate  all  of  the  complexities  of  a  classroom  environment  as  they  grapple  with 
how  to  arrange  the  conditions  and  experiences  to  best  promote  the  learning  of 
diverse  students. 

It  is  gratifying  to  see  that  most  of  these  novice  teachers*  cognitive  maps  and 
self-report  data  reveal  a  systematic  quality  to  their  planning  behavior.  The 
language  of  several  of  these  teachers,  but  certainly  not  all,  is  consistent  with  the 
principles  and  general  process  of  ISD.   It  would  be  desirable  to  conduct  a  more  fine- 
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gained  analysis  of  the  qualitative  differences  in  how  teachers  plan,  and  to  also 
search  for  relationships  between  teachers'  planning  and  student  outcome 

variables^  ^  ^  ^  interest  is  that  several  students  claimed  not  to  see  much 
benellt  from  having  studied  instructional  systems  design.  This  was  the  case  even 
for  some  students  whose  descriptions  of  their  own  behavior  indicated  that  they  were 
indeed  applying  a  basic  ISD  model  in  their  planning.  How  to  reconcile  the 
perception  that  ISD  principles  lack  utility  for  them  as  teachers  is  a  puzzle.  Perhaps 
some  students  confuse  form  with  process,  that  is,  they  come  to  see  an  ISD  model  and 
the  formal  planning  and  products  or  their  instructional  design  classes  as  the  form 
that  they  must  attempt  to  emulate  rather  than  a  process  that  can  guide  their  thinking 
and  planning  Tor  instruction.  , 

Reigeluth  (1983)  makes  a  distinction  between  instructional  design  theory 
and  learning  theory  when  he  states  that  the  former  "...must  include  specific 
Instructional  method  variables;"  and  is  therefore  relatively  easy  to  apply  in  the 
classroom.  On  the  other  hand  he  claims  that  learning  theory  is  typically  difficult 
to  apply  because  it  lacks  specificity  and  leaves  it  to  the  teacher  to  devise  the  specific 
applications  of  instruction.  I  believe  this  distinction,  while  somewhat  overstated, 
has  merit  in  aiding  our  understanding  of  why  some  teachers  reject  or  claim  to 
reject  their  training  in  instructional  design. 

We  need  to  acknowledge  and  help  preservice  teachers  understand  that  IbU 
can  be  a  valuable  tool  if  it  is  used  as  a  heuristic  for  planning  systematically,  rather 
than  being  perceived  as  an  absolute  and  mechanistic  formula  for  instructional 
planning  that  is  out  of  touch  with  the  realities  of  their  classrooms.  In  short,  we  must 
do  a  better  job  of  translating  and  modifying  the  essential  elements  of  the  ISD 
process  in  order  to  make  it  efficient  and  effective  for  teachers  at  all  levels,  and 
compatible  with  teachers'  perceptions  of  the  imperatives  of  teaching. 
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•Items  1,  2,  4,  9, 10, 12  and  18  are  starred  to  indicate  discrepancies  between  the 
means  of  the  two  groups  of  teachers.  A  significant  difference  (.05  level)  was  found 
for  item  4  (t=  2.704). 
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Rationale 

Traditionally,  audio  teleconferencing  has  been  a 
convenient  method  of  bringing  students  together  for 
distance  learning  (Wagner  &  Reddy,  1989) .     In  addition 
audio  teleconferencing,  some  colleges  and  universities 
have  bee*i  experimenting  with  more  sophisticated 
technologies  that  incorporate  computer  graphics 
(audiographic  teleconferencing) ;  full  motion  video 
distributed  via  satellite,  cable,  or  microwave;  and 
other  less  costly  audio/video  teleconferencing  systems 
such  as  slovj-scan  or  compressed  video  (Ritchie  &  Newby, 
1989;  Wagner  &  Reddy,  1989). 

Developers  of  more  sophisticated  teleconferencing 
systems  claim  that  these  systems  are  superior  to  the 
traditional  audio-conferencing  systems  because  they 
offer  the  added  benefit  of  visual  contact  among 
students  and  instructors  which  promotes  interaction 
(dialogue)  during  instruction.     The  research  upon  which 
this  claim  is  based  is  scarce,  however,  and  the  few 
studies  that  have  addressed  interaction  have  suffered 
from  poor  design. 

Problem 

Researchers  investigated  whether  there  was  a 
difference  in  either  the  overall  frequency  or  the 
frequency  of  specific  types  of  student  verbal 
interactions  among  the  following  three  groups: 

(a)  teleconferenced  instruction  where  students  had  two- 
way  audio  and  video  contact  with  the  instructor, 

(b)  teleconferenced  instruction  where  students  had  only 
two-way  audio  contact  with  the  instructor,  and 

(c)  traditional  face-to-face  instruction. 

Hypotheses 

This  study  was  designed  to  test  the  following 
hypotheses: 

1.  The  number  of  student  verbal  interactions  that 
occur  during  teleconferenced  instruction  where  students 
have  two-way  audio  and  video  contact  with  the 
instructor  will  be  greater  than  the  number  of  student 
verbal  interactions  that  occur  during  teleconferenced 
instruction  where  the  students  have  only  two-way  audio 
contact  with  the  instructor. 

2.  The  number  of  student  verbal  interactions  that 
occur  traditional  face-to-face  instruction  will  be 
greater  than  the  number  of  student  verbal  interactions 
that  occur  during  teleconferenced  instruction  where  the 
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students  have  only  two-way  audio  contact  with  the 
instructor. 

3.  The  number  of  student  verbal  interactions  that 
occur  during  traditional  face-to-face  instruction  will 
be  greater  than  the  number  of  student  verbal 
interactions  that  occur  during  teleconferenced 
instruction  where  the  students  have  two-way  audio  and 
video  contact  with  the  instructor. 

4.  The  number  of  restricted  thinking  questions 
that  students  ask  during  teleconferenced  instruction 
where  students  have  two-way  audio  and  video  contact 
with  the  instructor  will  be  greater  than  the  number  of 
restricted  thinking  questions  that  students  ask  during 
teleconferenced  instruction  where  the  students  have 
only  two-way  audio  contact  with  the  instructor. 

5.  The  number  of  restricted  thinking  questions 
that  students  ask  during  traditional  face-to-face 
instruction  will  be  greater  than  the  number  of 
restricted  thinking  questions  that  students  ask  during 
teleconferenced  instruction  where  the  students  have 
only  two-way  audio  contact  with  the  instructor. 

6.  The  number  of  restricted  thinking  questions 
that  students  ask  during  traditional  face-to-face 
instruction  will  be  greater  than  the  number  of 
restricted  thinking  questions  that  students  ask  during 
teleconferenced  instruction  where  the  students  have 
two-way  audio  and  video  contact  with  the  instructor. 

7.  The  number  of  expanded  thinking  questions  that 
students  ask  during  teleconferenced  instruction  where 
students  have  two-way  audio  and  video  contact  with  the 
instructor  will  be  greater  than  the  number  of  expanded 
thinking  questions  that  students  ask  during 
teleconferenced  instruction  where  the  students  have 
only  two-way  audio  ccntact  with  the  instructor. 

8.  The  number  of  expanded  thinking  questions  that 
students  ask  during  traditional  face-to-face 
instruction  will  be  greater  than  the  number  of  expanded 
thinking  questions  that  students  ask  during 
teleconferenced  instruction  where  the  students  have 
only  two-way  audio  contact  with  the  instructor. 

9.  The  number  of  expanded  thinking  questions  that 
stud    ts  ask  during  traditional  face-to-face 

inr  '  .action  will  be  greater  than  the  number  of  expanded 
thinking  questions  that  students  ask  during 
teleconferenced  instruction  where  the  students  have 
two-way  audio  and  video  contact  with  the  instructor. 

10.  The  number  of  restricted  thinking  responses 
that  students  give  during  teleconferenced  instruction 
where  students  have  two-way  audio  and  video  contact 
with  the  instructor  will  be  greater  than  the  number  of 
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restricted  thinking  responses  that  students  give  during 
teleconferenced  instruction  where  the  students  have 
only  two-way  audio  contact  with  the  instructor. 

11.  The  number  of  restricted  thinking  responses 
that  students  give  during  traditional  face-to-face 
instruction  will  be  greater  than  the  number  of 
restricted  thinking  responses  that  students  give  during 
teleconferenced  instruction  where  the  students  have 
only  two-way  audio  contact  with  the  instructor. 

12.  The  number  of  restricted  thinking  responses 
that  students  give  during  traditional  face-to-face 
instruction  will  be  greater  than  the  number  of 
restricted  thinking  responses  that  students  give  during 
teleconferenced  instruction  where  the  students  have 
two-way  audio  and  video  contact  with  the  instructor. 

13.  The  number  of  expanded  thinking  responses 
that  students  gWe  during  teleconferenced  instruction 
where  students  aave  two-way  audio  and  video  contact 
with  the  instructor  will  be  greater  than  the  number  of 
expanded  thinking  responses  that  students  give  during 
teleconferenced  instruction  where  the  students  have 
only  two-way  audio  contact  with  the  instructor. 

14 .  The  number  of  expanded  thinking  responses 
that  students  give  during  traditional  face-to-face 
instruction  will  be  greater  than  the  number  of  expanded 
thinking  responses  that  students  give  during 
teleconferenced  instruction  where  the  students  have 
only  two-way  audio  contact  with  the  instructor. 

15.  The  number  of  expanded  thinking  responses 
that  students  give  during  trad  tional  face-to-face 
instruction  will  be  greater  than  the  number  of  expanded 
thinking  responses  that  students  give  during 
teleconferenced  instruction  where  the  students  have 
two-way  audio  and  video  contact  with  the  instructor. 

Significance  of  the  Problem 

Most  of  the  research  in  distance  education 
consists  of  comparisons  between  traditional  on-campus 
courses  and  their  distance  counterparts  (Ritchie  & 
Newby,   1989) .     Verbal  interaction  is  rarely  reported  in 
these  intermedia  studies;  in  fact,  student/teacher 
verbal  interaction  is  often  not  permitted  during 
instruction  in  order  to  avoid  introducing  an 
uncontrollable  variable  (Salomon  &  Clark,  1977). 

Evidence  suggests  that  students'  perceived 
achievement  and  affect  towards  the  subject  and 
instructor  are  related  to  student/teacher  interaction. 
When  high  levels  of  verbal  interaction  are  present, 
students  report  that  they  learn  more  and  enjoy  the 
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experience  more  than  situations  that  involve  low  levels 
of  student/ teacher  interaction  (Richmond,  Gorham,  & 
McCroskey,   1987;  Gorham,  1988). 

During  audio  teleconferences,  nonverbal 
interaction  is  not  transmitted  between  remote  sites; 
however,  during  video  teleconferences  this  nonverbal 
information  is  transmitted.     Research  by  Wiener  and 
Mehrabian  (1968)  and  Mehrabian  (1971,  1972)  suggests 
that  some  of  this  nonverbal  information  has  the 
potential  of  inducing  verbal  interaction.     Since  verbal 
interaction  is  considered  to  be  desirable  during 
instruction,  one  could  make  a  case  for  developing  and 
utilizing  distance  delivery  systems  that  provide 
optimum  potential  for  two-way  student/teacher 
interaction.     In  order  to  do  so,  such  systems  would 
require  two-way  video  capabilities  in  order  to  transmit 
nonverbal  information  among  remote  sites.  These 
systems  are  expensive  and  require  significant  human 
resources  to  manage  and  maintain.     Institutions  must  be 
convinced  that  the  benefits  are  worth  the  costs  before 
investing  these  resources.     By  measuring  the  benefits 
in  terms  of  the  potential  for  student/teacher 
interaction  rather  than  exam  scores  and  course  grades, 
the  benefits  may,  indeed,  justify  the  costs. 

Methods  and  Procedures 

One-hundred  seventy-two  subjects  enrolled  in  four 
separate  sections  of  Classroom  Educational  Technology 
(ET  401),  during  the  Fall  semester,   1991  at  the 
University  of  Northern  Colorado  took  part  in  the  study. 
All  of  the  students  enrolled  in  ET  401  were 
Professional  Teacher  Education  (PTE)  students,  meaning 
that  they  were  pursuing  teacher  certification  in 
Colorado. 

There  were  four  separate  sections  of  ET  401,  each 
containing  about  forty-three  students.     Within  each 
section,  students  were  randomly  assigned  to  one  of 
three  treatment  groups.     In  other  words,  within  each  of 
the  four  sections  of  ET  401  there  were  three  treatment 
groups,  each  containing  about  fourteen  students.  After 
the  treatments  were  conducted,  the  results  of  the  four 
separate  sections  of  ET  401  were  collapsed  for  data 
analysis,  resulting  in  three  larger  treatment  groups  of 
approximately  fifty-seven  students  each. 

The  selected  site  for  this  study  was  the 
University  of  Northern  Colorado  (UNC)  College  of 
Education  and  College  of  Continuing  Education.  Two- 
way  audio  and  video  teleconferencing  facilities  exist 
between  the  Western  Institute  of  Distance  Education 
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(WIDE)  conference  room  and  Frasier  Hall,  both  located 
on  UNC's  Greeley  campus.    The  instructor  taught  from 
the  Frasier  Hall  site  and  the  students  were  located  in 
the  WIDE  conference  room  except  for  the  traditional 
face-to-face  treatment  where  the  instructor  was 
physically  present  in  the  WIDE  conference  room  with  the 
students.    Each  location  was  equipped  with  26" 
monitors,  conference  tables  and  seating  to  accommodate 
about  twenty  students,  plus  all  of  the  necessary 
hardware  and  software  in  order  to  operate  either  a  two- 
way  audio-only  teleconference,  or  a  two-way  audio  and 
video  teleconference. 

Treatment  Group  A  received  instruction  delivered 
via  two-way  audio  teleconference  (audio  group) . 
Treatment  Group  B  received  instruction  delivered  via 
two-way  audio-video  teleconference  (audio/video  group) . 
Treatment  Group  C  received  instruction  delivered  in  a 
traditional  face-to-face  manner  with  the  instructor 
present  in  the  WIDE  conference  room  with  the  students 
(traditional  group) . 

Each  group  of  fourteen  students  met  twice  in  the 
WIDE  conference  room  during  regular  class  hours.  The 
first  meeting  was  designed  to  desensitize  the  students 
to  the  newness  of  the  instructional  situation.     No  data 
were  collected  during  this  first  session.    The  first 
lesson  was  entitled  "Topics  in  Distance  Education." 
The  second  lesson  was  entitled  "Copyright  Issues  for 
Using  Videotapes  in  the  Classroom." 

The  formats  of  the  two  lessons  were  very  similar. 
The  instructor  gave  the  students  handouts  with  a  series 
of  true/false  questions  designed  to  stimulate  thinking 
on  the  subject.     Students  completed  these  questions  and 
the  instructor  went  over  the  answers.     The  instructor 
then  presented  the  new  information.    Students  were  told 
to  interrupt  if  they  had  questions  or  comments.  A 
series  of  open-ended  questions  and  short  case  studies 
were  then  presented.    The  instructor  read  the  question 
or  case  and  then  called  for  comments.    A  standard  wait 
time  of  five  seconds  was  used  during  all  instruction. 
If  students  did  not  respond,  the  instructor  would  issue 
one  more  call  for  responses  and  wait  an  additional  five 
seconds  before  moving  on  to  the  next  question  or  case. 
For  both  lessons,  students  were  given  handouts  with 
outlines,  discussion  questions,  and  cases. 

The  second  lesson  on  copyright  issues  was 
videotaped,  and  the  videotapes  were  reviewed  and  coded 
by  two  independent  consultants  according  to  the 
following  Equivalent  Talk  Categories  (ETC's): 
(a)  restricted  thinking  questions,   (b)  expanded 
thinking  questions,   (c)  restricted  thinking  responses, 
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and  (d)  expanded  thinking  responses  (Ober,  Bentley,  & 
Miller,   1971) . 

Results 

A  series  of  one-way  ANOVA's  and  Dunn's  t-tests 
were  conducted  for  each  of  the  equivalent  talk 
categories  to  determine  whether  there  were  difference.^ 
among  the  three  treatment  groups  (see  Tables  1-10) . 
The  following  summarizes  the  results  for  each  of  the 
equivalent  talk  categories. 

Total  Number  of  Student  Responses 

An  F  value  of  9.09  indicated  that  there  were 
significant  differences  among  the  three  groups  in  total 
number  of  student  responses.     The  difference  between 
the  means  of  the  audio  and  audio/video  groups  was  .0946 
(see  Table  2) .     This  difference  was  not  large  enough  to 
yield  significance  at  the  .05  level.     The  difference 
between  the  audio  and  traditional  groups,  however,  was 
2.62,  indicating  significance.    The  traditional  group 
interacted  more  than  the  audio  group  in  this  category. 

Dunn's  t-test  also  yielded  significance  between 
the  audio/video  group  and  the  traditional  group 
(difference  between  means  =  2.72).     The  traditional 
group  interacted  more  than  the  audio/video  group  in 
terms  of  the  total  number  of  interactions  given  during 
the  lesson  (see  Tables  1  and  2) . 

Table  1 

ANOVA  Summary  Table  for  Total  Number  of  Student 
Responses  by  Group. 


Source  df  SS  MS  F 


Between  Groups         2  270  135^  9 . 09** 

Within  Groups        169  2510  15 

Total  171  2781 

*p  <  .05  ~ 
**p  <  .01 
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Table  2 

Dunn's  Test  for  Total  Number  of  Responses  bv  Group 


♦Significant  at  .05 

Restricted  Thinking  Questions 

An  F  value  of  4.02  indicated  that  there  were 
significant  differences  among  the  three  groups  in  this 
category.     Dunn's  t-test  revealed  that  there  was  no 
difference  between  the  audio  group  and  the  audio/video 
group.     Also  there  was  no  difference  between  the  audio 
group  and  the  traditional  group.     Students  in  the 
traditional  group,  however,  initiated  more  restricted 
thinking  questions  than  students  in  the  audio/video 
group.     The  difference  between  the  group  means  was  .35, 
indicating  significance  at  the  .05  level   (see  Tables  3 
and  4) . 

Table  3 

ANOVA  Summary  Table  for  Restricted  Thinking  Questions 
bv  Group 


Source  df  SS  MS  F 


Between  Groups         2  3.41  1.71  4.02* 

Within  Groups        169  71.81  0.42 

Total  171  75.23 


*p  <  .05 
**p  <  .01 

Table  4 

Dunn's  Test  for  Restricted  Thinking  Questions  bv  Group 

G roup  Comparison  Difference  Between  Means 
A  &  B  0,0111 
A  &  C  0.2945 
B  &  C  0.3056* 


♦Significant  at  .05 


Group  Comparison 


Difference  Between  Means 


A  &  B 
fi  &  C 
B  &  C 


0.0946 

2.6216* 

2.7163* 
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Expanded  Thinking  Questions 

In  the  expanded  thinking  response  category, 
students  in  the  traditional  group  interacted 
significantly  more  than  students  in  the  video 
teleconference  group  (F  =  3.37)    (see  Table  6).  The 
audio  and  the  audio/video  groups  did  not  differ 
significantly,  with  differences  in  group  means  of  .13. 
The  difference  between  the  audio  and  traditional  group 
cf  .14  was  not  large  enough  to  indicate  significance  in 
this  category  (see  Tables  5  and  6) . 

Table  5 

ANOVA  Summary  Table  for  Expanded  Thinking  Questions  bv 
Group 

Source  df  ~SS  MS  F 
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Between  Groups         2  5.71  2,8  6  3.37* 

Within  Groups        169  143.28  0.85 

Total  171  149.00 

*p  <  .05 
**p  <  .01 

Table  6 

Dunn's  Test  for  Expanded  Thinking  Questions  bv  Group 

Group  Comparison  Difference  Between  Means 
A  &  B  0.1354 
A  &  C  0.1386 
B  &  C  0.4325* 

*Signif .icant  at  .05 

Restricted  Thinking  Responses 

A  large  F  value  of  10.35  showed  that  there  were 
significant  differences  among  the  three  groups  in  this 
category.    The  audio  group  gave  significantly  fewer 
restricted  thinking  responses  than  the  traditional 
group  (difference  between  means  =  1.19).  The 
audio/video  group  also  gave  significantly  fewer 
responses  than  the  traditional  group  (difference 
between  means  =  1.29).     No  significance  was  indicated 
between  the  audio  group  and  the  audio/video  group  in 
this  category  (see  Tables  7  and  8). 
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Table  7 

ANOVA  Summary  Table  for  Restricted  Thinking  Responses 
by  Group 


Source 

df                SS               MS  F 

Between  Groups 
Within  Groups 
Total 

2                59.15          29.57  10.35** 
169              482.97  2.86 
171  542.12 

*p  <  .05 
**p  <  .01 

Table  8 

Dunn's  Test  for  Restricted  Thinkina  Responses  by  Group 

Group  Comparison 
A  &  B 
A  &  C 
B  &  C 

Difference  Between  Means 
0.1039 
1.1917* 
1.2956* 

♦Significant  at  .05 


Expanded  Thinking  Responses 

The  final  category  tested  was  expanded  thinking 
responses,  and  an  F  value  of  3.55  indicated 
significance  at  the  .05  level  among  the  three  groups. 
Dunn's  t-test  showed  that  there  was  a  significance 
difference  between  the  audio  group  and  the  traditional 
group — the  traditional  group  gave  more  expanded 
thinking  responses  than  the  audio  group  (difference 
between  means  =  .80).    No  differences  were  indicated 
between  the  audio  and  audio/video  groups,  nor  between 
the  video  and  traditional  groups  (sec  Tables  9  and  10) . 

fable  9 

ANOVA  Summary  Table  for  Expanded  Thinking  Responses  by 
Group 


Source  df  SS  MS  F 


Between  Groups  2  20.88  10.44  3.55* 

Within  Groups        169  496.77  2.94 

Total  171  517.65 


*p  <  .05 
**p  <  .01 
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Table  10 

Dunn's  Test  for  Expanded  Thinking  Responses  bv  Group 


*Significant  at  .05 

Implications  and  Recommendations  for  Further  Study 

The  results  of  this  study  indicated  that  students 
were  not  likely  to  interact  more  during  teleconferenced 
instruction  where  stude  its  had  two-way  audio  and  video 
contact  with  the  instructor  than  they  would  during 
teleconferenced  instruction  where  students  had  only 
two-way  audio  contact  with  the  instructor.    As  a  result 
Hypothesis  1  was  rejected  at  the  .05  level.  The 
results  indicated  that  students  were  likely  to  interact 
more  during  traditional  face-to-face  instruction  than 
they  would  during  either  an  audio  only  or  an 
audio/video  teleconference.     These  findings  were 
consistent  with  Hypotheses  2  and  3. 

In  the  specific  Equivalent  Talk  Categories 
(ETC's),  the  control  group  asked  significantly  more 
questions — both  in  the  restricted  and  expanded  thinking 
categories — than  the  audio/video  group.    These  findings 
were  consistent  with  Hypotheses  6  and  9.      The  audio 
group  and  the  traditional  group,  however,  did  not 
differ  significantly  in  these  ETC's.    As  a  result, 
Hypotheses  4  and  7  were  rejected  at  the  .05  level. 
Also,  there  were  no  significant  differences  between  the 
audio  and  the  traditional  group  in  either  restricted  or 
expanded  thinking  questions;  consequently,  Hypotheses  5 
and  9  were  also  rejected  at  the  .05  level.  Further 
research  needs  to  be  conducted  to  determine  exactly  why 
the  predictions  given  in  Hypotheses  4,  5,  7,  and  8  did 
not  materialize. 

Both  the  audio  and  the  audio/video  groups  gave 
significantly  fewer  restricted  thinking  responses  than 
the  control  group.    This  was  consistent  with  Hypotheses 
11  and  12.    The  audio  and  the  audio/video  groups, 
however,  did  not  differ  significantly  in  this  ETC  at 
the  .05  level  (reject  Hypothesis  10).     More  research  is 
needed  to  explain  this  finding. 

In  the  final  ETC,  expanded  thinking  questions, 
there  were  no  significant  differences  between  the  audio 
and  the  audio  video  group,  or  between  the  audio  and  the 


Group  Comparison 


Difference  Between  Means 


A  &  B 
A  &  C 
B  &  C 


0.1180 

0.8005* 

0.6825 
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traditional  group.     As  a  result,  Hypotheses  13  and  15 
were  rejected  at  the  .05  level.     Further  research  is 
needed  to  explain  these  findings.     There  Were 
significant  differences  between  the  traditional  group 
and  the  audio/video  group  (Hypothesis  15) . 

In  general,  two  patterns  emerged  from  this  study. 
First,  the  audio  and  audio/video  groups  did  not  differ 
significantly  in  any  of  the  ETC's,  nor  did  they  differ 
in  terms  of  the  total  number  of  interactions  during  the 
teleconferenced  instruction.    The  question  that  needs 
to  be  answered  is,  "Can  nonverbal  cues  that  induce 
interaction  be  transmitted  via  television?"  Perhaps 
the  explanation  lies  in  the  way  students  are 
conditioned  to  watching  a  television  screen.  Viewing 
television  is  a  passive  activity  and  viewers'  cognitive 
engagement  appears  to  wane  over  time  (Brown,  1988). 
The  students  in  the  audio/video  group  in  this  study  may 
not  have  been  mentally  focused  on  the  instructor  when 
the  open-ended  questions  were  presented  at  the  end  of 
the  lesson.    As  a  result  of  this  lack  of  attention,  the 
audio/video  teleconference  experience  was  basically  the 
same  as  the  audio  teleconference  experience. 

The  second  pattern  that  emerged  was  that  the 
traditional  group  generally  interacted  more  than  the 
audio  and  the  audio/video  groups  in  terms  of  total 
number  of  interactions  and  in  several  of  the  ETC's. 
The  implications  of  this  finding  on  distance  education 
are  that  occasional  site  visits  on  the  part  of  the 
instructor  may  be  beneficial.    Further  research  could 
be  conducted  to  determine  whether  site  visits  increase 
the  amount  of  interaction  that  occurs  during 
teleconferences  that  follow  the  site  visits. 
Researchers  in  this  study  were  only  interested  in  the 
effects  that  student/ instructor  visual  contact  had  on 
verbal  interaction  during  teleconferences.  The 
audio/video  capabilities  of  the  system  used  for  the 
two-way  audio/video  teleconference  would  allow  the  user 
to  take  advantage  of  the  attributes  inherent  in  such  a 
system  by  sharing  graphics  and  other  visual 
information.    This  may  in  turn  stimulate  verbal 
interaction.     Further  research  needs  to  be  conducted  in 
order  to  determine  whether  effective  uses  of  the  video 
capabilities  of  teleconferencing  systems  influences  the 
amount  of  interaction  that  takes  place  during 
instruction. 

Finally,  the  assumption  that  the  addition  of 
visual  contact  is  in  and  of  itself  capable  of  improving 
distance  education  is  not  substantiated  in  this  study. 
In  future  studies,  it  might  be  beneficial  to  measure 
something  other  than  the  amount  of  verbal  interaction 
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that  takes  place  during  teleconferences  and  focus  on 
the  role  of  nonverbal  interactions  in  visual  delivery 
modes. 
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Papers 

Introduction  to  Thinking  Otherwise: 
Critical  Theory,  Ethics  and  Postmodernism 

Dr.  Andrew  R.  J.  Yeaman,  Yeaman  &  Associates,  Denver,  Colorado 

The  imperative  for  many  cultural  agents  in  the  era  of  concentration 
camps  and  nuclear  weapons  has  been  to  think  "otherwise,"  to  transgress 
the  coherent  unity  of  a  metaphysics  that  has  proven  inadequate  to  the 
problems  we  face. 

F  (Ulmer,  1986,  p.  27) 

The  purpose  of  this  symposium  is  to  examine  the  ethics  of  educational 
communications  and  technology.  The  essayists  writing  here  reflect  on  ethics 
through  contemporary  critical  theorists  such  as  Derrida,  Foucault  and 
Habermas  among  others,  itie  critical  theory  approach  is  philosophical,  literary 
and  sociopolitical.  The  emphasis  is  not  on  the  ethical  behavior  of  individuals, 
which  seems  to  be  the  domain  of  the  AECT  Code  of  Ethics,  but  on  the  ethical 
position  of  educational  communications  and  technology  in  society. 

A  secondary  purpose  is  to  encourage  more  scholars  to  engage  in  cultural 
analysis.  The  authors  address  ethical  aspects  of  the  field  and  provide  insight 
by  demonstrating  the  application  of  critical  theory.  This  symposium  is 
intenued  to  stimulate  questions  about  critical  theory  and  will  encourage 
discussion  about  the  global  effects  of  the  field. 

What  is  critical  theory? 

Critical  theory  dates  back  at  least  to  Socrates  privileging  the  reality  ot 
ideas  or  forms  over  the  reality  of  appearances  (Adams,  1971).  Over  the 
millennia  this  debate  on  interpretation,  representation  and  media  became  the 
intellectual  foundation  of  philosophy,  theology,  literary  criticism,  linguistics, 
psychology,  communications  and  education.  In  the  1960s  the  l'terary  focus  of 
critical  theory  in  North  America  was  augmented  through  importing  the 
European  emphasis  on  sociological  analysis  and  the  crit"  ^ue  of  ideology 
(Adams  &  Searle,  1986). 

There  is  no  unified  critical  theory  but  critical  thought1  is  nevertheless 
influential.  Despite  their  disagreements,  critical  theorists  tend  to  concur  that 

1  Critical  theory  should  not  be  conflated  with  critical  thinking,  the  recent 
pedagogical  movement  advocating  teacher  centered  questioning  for  the  development 
of  thinking  skills. 
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reality  is  socially  constructed,  that  the  positr/istic  labeling  of  people  is  not 
natural  and  that  scientific  explanations  of  human  behavior  lack  objectivity 
(Gibson,  1986).  Contemporary  texts  produced  by  critical  theorizing  are 
viewed  not  as  derivative,  secondhand  literature  but  as  original,  primary  works 
actively  confronting  real  problems  (Eagleton,  1983;  Hartman,  1981).  Current 
scholarship  in  education  takes  this  point  of  view,  especially  Critical  theory 
and  education  (Gibson,  1986),  Postmodern  education:  Politics,  culture,  and 
social  criticism  (Aronowitz  &  Giroux,  1991),  Postmodernism,  feminism,  and 
cultural  politics  (Giroux,  1991)  and  Power  and  criticism:  Poststructural 
investigations  in  education  (Cherryholmes,  1988).  Direct  application  to 
educational  communications  and  technology  is  made  by  Paradigms 
regained:  The  uses  of  illuminative,  semiotic  and  post-modern  criticism  as 
modes  of  inquiry  in  educational  technology  (Hlynka  &  Belland,  1991)  and 
The  ideology  of  images  in  educational  media  (Ellsworth  &  Whatley,  1990). 

This  introduction  also  suggests  these  questions: 

•  How  is  criticism  related  to  education? 

•  What  are  poststructuralism  and  postmodernism? 

•  How  is  critical  theory  related  to  ethics? 

•  What  are  postmodern  ethics? 

•  How  are  postmodern  ethics  related  to  educational  technology? 

The  critical  theorists  referenced  in  this  symposium  diverge  from  each 
other  but  issues  of  unity  or  fragmentation  are  exactly  not  the  point.  The 
questions  applied  here  to  the  field  of  educational  communications  and 
technology  are  important  to  raise.  The  group  writing  in  these  pages  seeks  to 
develop  understandings  of  ethical  conscience  as  the  prerequisite  to  any  action. 
Whether  or  not  the  discourse  of  any  emancipatory  metanarrative  is  true,  for 
critical  theorists  the  ethical  approach  is  necessary. 

Deconstruct  ion  and  Educational  Media 

Dr.  Andrew  R.  J.  Yeaman,  Yeaman  &  Associates,  Denver,  Colorado 

Derrida  is  part  of  that  general  movement  in  the  humanities  of  our 
century,  responding  in  protest  to  the  ideological  constraints  of  a 
civilization  that  seems  bent  on  self-destruction. 

(Ulmer,  1986,  p.  27) 

Deconstruction  can  reveal  distortions  and  biases  in  education 
(Cherryholmes,  1988)  and  educational  technology  (Hlynka  &  Belland,  1991). 
For  example,  this  presentation  demonstrates  how  a  safety  message 
deconstructs  into  the  concerns  of  manufacturers  and  dealers  about  product 
liability.  It  shows  that  doublespeak  obscures  the  power  relations  of  the  social 
context  (Bourdieu  &  Passeron,  1990): 
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How  do  things  slip  around  so  that  the  accidents  caused  by  badly  designed 
media  equipment  are  no  longer  the  responsibility  of  manufacturers, 
suppliers  and  purchasing  officers  but  become  the  fault  of  instructors? 

Reading  deconstructively  as  a  way  of  identifying  blind  spots  is  originally 
described  by  de  Man  (1983)  and  Derrida  (1973,  1976,  1985,  1987a,  1990)  and 
has  profoundly  affected  literary  interpretation  and  analytic  philosophy  (Rorty, 
1989).  Norris  explains,  "To  'deconstruct'  a  text  is  to  draw  out  conflicting 
logics  of  sense  and  implication,  with  the  object  of  showing  that  the  text  never 
exactly  means  what  it  says  or  says  what  it  means."  (1988,  p.  7). 

Deconstruction  translates  to  painting  (Derrida,  1987b),  architecture 
(Norris  &  Benjamin,  1988;  Papadakis,  Cooke  &  Benjamin,  1990),  video 
(Ulmer,  1989)  and  cinema  (Brunette  &  Wills,  1989).  Deconstruction  is  also 
useful  when  considering  the  ethics  of  designing,  selecting  and  evaluating 
educational  media. 

Restructuring,  Technology  and  Schools 

Dr.  Robert  Muffoletto,  University  of  Northern  Iowa,  Cedar  Falls,  Iowa 

This  paper  inquires  into  the  culture  of  schools  restructuring  and  the  role 
of  technology  (Bowers,  1988).  California  and  Iowa  function  as  platforms  for 
observation  and  analysis.  The  critique  will  unpack  the  rationalizations, 
justifications  and  envisioned  results  from  the  literature  of  each  locale. 

Constructing  and  reconstructing  schools  are  social  political  processes. 
American  schools  serve  purposes  reflected  in  the  symbolic  interactions, 
regulations,  and  controls  over  what  teachers,  administrators,  students  do  and 
think,  plus  the  form  and  content  of  knowledge  disseminated  and  reproduced 
daily  in  classrooms  (Apple,  1982;  Apple,  1988;  Apple  &  Weis,  1983, 
Gerbner,  1974,  Kliebard,  1986).  The  conceptualization  and  use  of  various 
technologies  became  over  the  last  50  years  a  major  part  of  schooling  (Cuban, 
1986;  Saettler,  1990).  Restructuring,  and  the  role  technology  plays,  can  be 
better  understood  when  contextualized  within  the  historical  struggles  for 
control  of  American  education.  Within  this  context  the  unpacking  of  the 
symbolic  may  reveal  a  different  agenda. 

These  postmodern  questions  (Cherryholmes,  1988)  will  guide  the  inquiry: 

•  What  are  the  historical  assumptions  about  education,  technology,  and  the 
larger  state  agenda  that  guide  the  restructuring  process? 

•  What  will  change  because  of  the  new  order? 

•  Who  makes  the  decisions  and  sets  the  agenda  for  change? 

•  Who  may  benefit  from  the  change  if  it  is  carried  out  as  prescribed? 
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The  Rite  of  Right  or  The  Rigjit  of  Rite: 
Moving  Towards  an  Ethics  of  Technological  Empowerment 

Ms.  Jane  Anderson,  Ohio  State  University,  Columbus.  Ohio 

From  Freire's  work  in  emancipatory  pedagogy  emerge  assumptions  of 
self-determination,  collective  identification,  and  reorientation  to  positions  of 
power  (1970,  1978,  1985).  What  do  these  assumptions  suggest  to  the  ethics  of 
the  liberatory  researcher  (Shor,  1987)  involved  in  the  promotion  of 
educational  technology?  In  framing  a  liberatory  approach  to  educational 
technology,  I  focus  on  the  human  right  and  the  cultural  rite.  Foucauldian 
notions  on  the  concept  of  power-knowledge  and  ths  role  of  the  "specific 
intellectual"  will  be  addressed  due  to  their  affinity  to  these  concepts. 

My  discussion  considers: 

•  What  might  be  an  ethics  of  technological  and  communicative 
empowerment  after  considering  Freire's  emancipatory  pedagogy  and 
Foucault's  notions  of  injustice  and  oppression? 

•  How  can  our  field  foster  the  creation  of  spaces  where  a  multiplicity  of 
views  can  co-exist  and  support  each  other? 

•  What  approach  should  be  taken  to  promote  technological  skills  among 
people  seeking  a  greater  voice? 

Feminisms,  Foucault  and  Felicitous  Design 

Dr.  Suzanne  K.  Damarin,  Ohio  State  University,  Columbus,  Ohio 

Some  feminist  philosophers  (e.g.,  Noddings,  1984;  1989)  and  social 
scientists  (Gilligan,  1982;  Belenky,  Clinchy,  Goldberger  &  Tarule,  1986) 
suggest  that  an  ethic  of  care  should  guide  instructional  activity  and,  by 
inference,  the  design  of  instruction.  The  ethic  of  caring  is  detailed  by  these 
writers  in  sharp  contrast  with  the  ethics  of  justice  which  undergird  most 
discussions  of  ethics  in  relation  both  to  technology  (e.g.,  Ermann,  Williams  & 
Gutierrez,  1990)  and  to  educational  issues  such  as  equity.  Discussing  an  ethic 
of  caring  thereby  introduces  new  (feminist)  ideas  into  this  field. 

Further,  the  postmodern  feminist  critique  rejects  the  ethic  of  caring  as 
essential izing.  Foucauldian  analyses  reveal  that  "caring"  (like  justice) 
promotes  hierarchal  observation,  normalizing  judgments,  etc.  (Foucault, 
1979).  These  observations  problematize  the  implications  of  an  ethic  of  care 
and  raise  issues,  questions  and  suggestions  about  the  design  of  instruction.  , 
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Communicative  Action  and  Educational  Biotechnology 

Dr.  Randall  G.  Nichols,  University  of  Cincinnati,  Cincinnati,  Ohio 

Habermas'  studies  conclude  that  growing  uses  of  media,  and  the  rational- 
instrumental  thinking  that  accompanies  media,  tends  to  remove  humans  from 
meaningful  communication  and  community  (1984,  1987).  This  growth  and 
removal  inherently  involves  questions  of  ethics.  Nichols  extends  this 
argument  to  the  educational  uses  of  both  media  and  extremely  rational 
thinking  (1989,  1991).  Building  on  that  foundation,  this  paper  looks  tc  a  likely 
near  future  of  educational  technology  (Knirk  &  Gustafson,  1986;  Perelman, 
1990)  and  applies  Habermas'  theory  of  communicative  action.  Specifically, 
educational  biotechnology  (EBT)  will  appear  in  the  daily  lives  of  all  educators 
and  learners  (Heinich,  Molenda  &  Russell,  1989,  p.  403)  because  of  the 
proliferation  of  genetic  mapping  and  manipulation,  the  altering  of  human 
chemistry  and  public  and  private  pressures  to  improve  American/Western 
educational  outcomes  (Noble,  1989).  The  paper  will  suggest  the  use  of 
Habermas'  moral  principles  as  guidelines  for  managing  the  ethical  and 
educational  dimensions  of  this  near  future. 

Marginalizing  Significant  Others: 
The  Canadian  Contribution  to  Educational  Technology 

Dr.  Denis  Hlynka,  University  of  Manitoba,  Winnipeg,  Canada 

Postmodernists  have  developed  a  variety  of  concepts  and  strategies  for 
examining  contemporary  society.  For  example,  the  Derridian  concept  of 
supplementation  of  the  dominant  discourse  with  minority  discourses  has  met 
with  significant  change  in  practice.  Black  studies,  feminist  praxis,  and  ethnic 
discourses  are  only  three  such  developments.  Eco's  exploration  of  literary 
texts  as  open  and  closed  is  reflected  in  slippery  signifiers.  Marginalization  and 
deconstructionhave  resulted  in  an  interchange  of  dominant  and  colonized. 
Baudrillard's  precession  of  the  simulacrum  has  resulted  in  a  re-thinking  of 
origins.  Finally,  Lyotard's  postmodern  condition  has  allowed  us  new  ways  of 
looking  at  the  information  revolution. 

For  several  of  these  writers  and  philosophers  technology  is  an  integral 
component  of  their  thii  king  and  there  are  serious  technological  implications 
to  be  derived  from  a  postmodern  perspective.  But  educational  technology  is 
only  beginning  to  examine  such  a  perspective. 

Canadian  educational  technology  offers  a  clear  case  of  postmodernism  in 
action.  Because  of  the  peculiarities  of  size  ard  populat  ion  (Canada  is  second 
in  geographic  size  next  to  Russia,  yet  with  a  population  smaller  than  the  state 
of  California)  and  because  of  the  Canadian  proximity  to  an  economic  giant 
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(the  United  States  of  America),  Canada  has  long  exhibited  schizophrenic 
behavior.  In  many  ways,  to  be  Canadian  is  to  be  postmodern. 

First,  Canada  as  a  whole  is  marginalized.  It  exists  under  the  shadow  of  its 
neighbor.  Its  culture  is  uniquely  Canadian,  yet  you  can  hardly  find  that  culture 
by  watching  Canadian  television  or  looking  in  the  shelves  of  Canadian 
bookstores.  Canada  is  a  minority  discourse,  struggling  for  its  own  place  in  the 
world.  In  educational  technology,  Canada  has  unknowingly  provided 
consistent  but  marginal  contributions  which  have  first  deferred  to  the 
overwhelmingly  American  contribution,  then  differed  from  those 
contributions,  and  then  finally  supplemented  the  American  discourse  on 
educational  technology.  What  is  the  Canadian  discourse  on  educational 
technology?  In  brief  it  consists  of  the  following: 

•  Marshall  McLuhan,  a  postmodernist  before  his  time,  predated  Derrida 
with  his  Laws  of  Media  and  provided  the  foundation  for  Baudrillard's 
exploration  of  the  simulacrum  with  his  famous  one-linen  The  medium  is  the 
message. 

•  The  National  Film  Board  of  Canada,  a  unique  production  unit  which, 
under  the  guidance  of  John  Grierson  gave  meaning  to  the  term  documentary 
and  provided  a  model  which  was  adopted  and  then  adapted  by  nearly  all  major 
educational  film  and  television  production  organizations. 

•  Arthur  Kroker,  Canadian  postmodern  political  scientist  has  examined 
technology  and  the  Canadian  mind  by  showing  a  continuum  between 
despondency  and  freedom,  and  an  intermixing  of  contributions  of  art  and 
technology.  The  well  known  technologists  who  form  the  crux  of  Kroker's 
exposition  are  George  Grant,  Harold  Innis  and  Marshall  McLuhan. 

•  Ursula  Franklin,  Canadian  physicist,  has  recently  contributed  to  the 
debate  by  showing  how  the  real  world  of  technology  is  mediated  by  social, 
political  and  cultural  factors  which  must  be  kept  up  front  if  one  is  to  avoid 
being  crushed  by  the  inexorable  rush  of  technology  out  of  control. 

The  twenty  first  century  is  about  to  begin  on  an  ambiguous  and  decentered 
note.  Educational  technologists  know  that  the  answer  to  an  improved 
education  is  not  merely  more  technology,  nor  faster  technology,  nor  even 
more  accurate  technology.  Educational  technologists  however  are  not 
Luddites  who  want  less  technology,  slower  technology  or  even  no  technology. 
Canada  has  made  major,  albeit  often  unrecognized  and  even  unnoticed 
contributions  to  the  dialogue  exploring  these  relationships  between  technology 
and  education.  The  purpose  of  this  paper  is  to  put  that  record  in  its  proper 
perspective. 
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Discussion 

Commentary  on  Postmodern  Implications  for  the  Future  of 
Theory,  Research  and  Development 

Dr.  J.  Randall  Koetting,  University  of  Nevada-Reno,  Reno,  Nevada 

The  commentary  provides  an  overview  of  the  contributions  and  compare 
their  positions  on  the  ethics  of  the  field.  Observations  are  made  about  the 
future  direction  of  theory  and  research.  The  commentary  includes  the 
application  of  theory  and  research  to  postmodern  development  in  the  field. 
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The  Effects  of  Two  Instructional  Conditions  on  Learners1  Computer  Anxiety  and 

Confidence. 

In  spite  of  the  diffusion  of  computers  into  our  society ,  there  remains  a  large  number  of 
people  who  continue  to  suffer  from  computer  anxiety.  Zelman  (1986)  reported  the  results 
of  a  1985  survey  of  over  2  million  school  teachers  by  the  Corporation  of  Public 
Broadcasting  in  which  31%  of  the  respondents  felt  uncomfortable  using  computers  and 
13%  reported  avoiding  computers.  Computer  anxiety  appears  to  be  a  problem  for  a  large 
number  of  people. 

Computer  anxiety  is  generally  viewed  as  a  form  of  state  anxiety  associated  with 
computer  use  (Howard,  1984;  Cambre  &  Cook,  1985)  and  is  usually  defined  in  a  manner 
such  as  "the  fear  or  apprehension  felt  by  individuals  when  they  used  computers,  or  when 
they  considered  the  possibility  of  computer  utilization11  (Simonson,  Mauer,  Torardi,  & 
Whitaker,  1987,  p.  238).  This  fear  or  apprehension  is  generally  believed  to  cause 
debilitating  thoughts.  These  thoughts  negatively  affect  learning  by  interfering  with  the 
processing  and  encoding  of  information  during  instruction. 

As  our  society  becomes  more  dependent  upon  retrieval  of  information,  computer 
technology  permeates  our  organizational  structures.  Nearly  half  of  all  white-collar 
workers  are  using  computers  in  their  daily  work  (Howard,  Murphy,  &  Thomas,  1987). 
Sanders  and  Stone  (1986)  report  that  the  U.  S.  Department  of  Labor  estimates  that  for  this 
next  generation  as  much  as  75%  of  all  jobs  will  involve  the  use  of  computers.  This 
importance  of  computer-related  skills  for  future  career  requirements,  makes  the  issue  of 
computer  anxiety  a  contemporary  issue  in  education.  Because  of  the  size  and  the 
importance  of  this  problem,  we  need  to  analyze  the  results  of  research  on  computer 
anxiety  so  that  we  can  more  se/iously  address  learners  suffering  from  the  debilitating 
effects  of  computer  anxiety. 

Most  of  the  literature  on  computer  anxiety  involves  case  studies.  These  articles 
relate  experiences  involving  teaching  of  workshops  or  courses  with  computers  and 
reflections  of  the  authors  perceptions  about  what  works  or  does  not  work.  While  these 
articles  may  be  helpful  to  those  designing  and  delivering  computer  instruction,  they 
contribute  very  little  to  the  development  of  a  research  base  in  computer  anxiety. 

Howard,  Murphy,  and  Thomas  (1987)  state  that  future  research  needs  to  examine  the 
types  of  instructional  strategies  that  are  most  effective  in  reducing  computer  anxiety. 
Tobias  (1979)  suggests  that  it  is  important  to  investigate  the  aptitude-treatment- 
interactions  in  areas  of  anxiety,  in  order  that  we  can  better  fit  anxiety  reduction 
strategies  to  the  individual  characteristics  of  learners. 

Purpose  of  the  Study 

Because  the  literature  is  lacking  research  that  investigates  the  influence  of  specific 
instructional  strategies  on  the  reduction  of  computer  anxiety  for  learners,  this  need 
should  be  addressed.  In  view  of  the  importance  of  computer  anxiety  and  attitudes  toward 
computers  to  the  instruction  and  training  of  learners,  the  purpose  of  this  study,  therefore, 
was  to  investigate  the  relationship  of  computer  anxiety  reduction  to  instructional 
strategies  and  learner  characteristics. 

Specifically,  this  study  sought  answer*  to  the  following  broad  questions: 

1.  What  type  of  instructional  intervention  contributes  to  reduced  computer  anxiety? 

2.  What  learner  characteristics  interact  with  the  types  of  instructional 
interventions  that  reduce  computer  anxiety? 

3.  What  is  the  nature  of  the  learners1  perceptions  that  occur  during  instructional 
reduction  of  computer  anxiety? 

Anxiety 

In  spite  of  a  trend  toward  increased  research  in  the  area  of  human  anxiety,  anxiety  as 
a  construct  has  still  resisted  a  consensus  of  its  definition  and  measurement  (Cambre  & 

041 


The  Effects  of  Two  Instructional  Conditions 


Page  3 


Cook,  1985).  There  is,  however,  a  commonality  to  most  definitions  of  anxiety— "a  fear 
about  something  in  the  future"  (Howard  &  Smith,  1986).  Cattell  &  Scheier  (1958)  using 
factor  analytic  studies  developed  the  concepts  of  two  types  of  anxiety.  The  first,  called 
trait  anxiety,  is  a  generalized  anxiety  or  a  global,  basic,  and  permanent  tendency  to  be 
anxious.  The  second,  state  anxiety,  is  a  situation  specific,  fluctuating,  and  transitory 
tendency  to  become  anxious, 

Eysenck  (1979)  and  Tobias  (1979)  have  suggested  that  anxiety  is  particularly 
debilitating  to  learning  when  the  task  is  difficult  and  when  there  is  a  strong  need  for 
memory  in  performance  of  the  task.  It  is,  therefore,  important  to  intervene  and  facilitate 
the  reduction  of  anxiety  when  its  effects  are  impairing. 

Tobias  (1979)  proposed  a  model  for  the  effect  of  anxiety  on  learning  from  instruction. 
This  model  suggested  that  anxiety  interferes  most  with  learning  before,  and  during  input 
by  the  learner.  Before  processing,  anxiety  acts  as  a  diversion  to  attention.  During 
processing,  anxiety  directly  interferes  with  the  cognitive  processing  and  storage  of 
information  by  the  learner.  Postprocessing  anxiety  obstructs  later  retrieval  of  content 
mastered  during  instruction.  Based  on  this  model,  Tobias  made  recommendations 
regarding  instruction  ^hich  is  expected  to  reduce  the  effects  of  anxiety  on  learning.  He 
suggested  that  instruction  for  anxious  learners  should  allow  learners  to  repeat  content 
and  reduce  the  extent  to  which  learners  must  rely  on  memory. 

M.  W.  Eysenck  (1984)  stated  that  actual  performance  for  high  anxiety  individuals  and 
low  anxiety  individuals  is  less  clear  than  the  anxiety  differences.  He  stated  that 
experimental  evidence  suggests  that  high  anxiety  learners  compensate  for  decreased 
cognitive  capacities  (due  to  anxiety  related  task-  irrelevant  cognitive  activities)  by 
increasing  effort  on  the  task.  Anxiety  level,  itself,  is  not  the  determining  factor  on 
behavior  and  choice.  It  is  necessary,  therefore,  to  look  at  the  interaction  of  anxiety 
stimuli  and  specific  characteristics  of  the  learners,  which  might  help  predict  or  interpret 
their  behaviors  under  conditions  of  anxiety. 

Activation  Theory 

Activation  Theory  (Berlyne,  1960;  Fiske  &  Maddi,  19151;  Malmo,  1971)  as  it  applies  to 
anxiety  has  typically  involved  three  concepts:  impact,  activation,  and  arousal.  Impact 
can  be  thought  to  be  the  momentary  contribution  of  a  stimulus  to  the  reticular  arousal 
system  (RAS),  a  network  of  nerve  cells  that  extends  through  the  lower  brain.  Activation 
is  the  degree  of  physiological  excitation  in  the  RAS.  Activation  level  is  related  to  the 
intensity  of  the  impact,  but  can  vary  due  to  the  influence  of  changes  in  the  somatic,  or 
physical,  state  of  the  individual.  Arousal  is  the  actual  manifestation  of  the  activation  in 
the  individual,  such  as  the  emotions  of  anxiety  or  anger.  Arousal,  in  turn,  affects 
activation  through  feedback,  by  changing  the  impact  of  other  stimuli. 

The  relationship  of  performance  to  activation  and  arousal  (Fiske  &  Maddi,  1961; 
Malmo,  1971)  has  been  summarized  in  three  statements:  (a)  there  is  an  optimal  level  of 
arousal  for  best  performance;  (b)  above  or  below  this  optimal  level,  performance  is 
relatively  impaired;  and  (c)  performance  impairment  increases  with  the  distance  fum 
the  optimal  level.  Studies  (Hines  &  Mehrabian,  1979;  Mehrabian  &  West,  1977;  Russell 
&  Mehrabian,  1975)  have  shown  a  detrimental  effect  of  relatively  high  levels  of  arousal 
on  performance  and  attitude  in  work  situations.  Russell  and  Mehrabian  (1975)  also 
found  that  in  unpleasant  situations,  avoidance  behavior  is  a  direct  correlate  to  arousal 
state.  The  optimal  level  of  arousal  varies  from  individual  to  individual. 

Arousal -Seeking  Tendency 

Each  individual  has  an  optimal  level  of  arousal,  not  only  for  performance,  but  also  for 
emotional  comfort.  A  person's  preferred  arousal  level  (Mehrabian  &  Russell,  1974)  is 
closely  related  to  his  or  her  preference  for  an  environment.  The  avoidance  or  attraction 
of  individuals  for  a  stimuli,  such  as  a  computer,  is  related  to  not  only  their  level  of 
arousal,  such  as  anxiety,  but  also  to  their  preferred  level  of  arousal.  Mehrabian  and 
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Russell  (1974)  have  isolated  a  hor.cgenous  trait  they  call  "arousal-seeking  tendency" 
which  is  a  measure  of  that  level  cf arousal  that  individuals  find  most  comfortable.  The 
arousal-seeking  tendency  was  defined  as  the  extent  of  the  level  of  arousal  fiat  an 
individual  finds  most  cr-jifcrteble. 

Mehrabian  &  Kr-sse'l  (1974)  developed  the  Arousal-Seeking  Tendency  Scale  (ASTS). 
They  administered  the  ASTS  «>,d  other  Inventories  to  530  subjects.  Results  showed  that 
ASTS  scorsf  were  signifies t<y  correlated  to  state  anxiety,  trait  anxiety,  and 
extrovert* un.  Mehralian  \         using  325  college  undergraduates,  found  other 
significant  correlates  co      ASTS*  Sensitivity  to  rejection  was  negatively  and 
mr  lerat  \y  correlate  *o  ASTS  scares.  Achieving  tendency,  extrovertism,  and 
oommr  nce  were  moderately  positively  correlated  to  ASTS  scores. 

Mdirahfrn  (1978),  in  a  later  soody  of  118  undergraduates,  investigated  individual 
react  •  n*f  to  positive  anr*  negative  situations.  Results  showed  that  high  arousal  seekers 
shov  ed  a  significant1;/  greats  approach  tendency  for  preferred  environments  than  low 
aroxsal  seekers.  Thiv  r*fr«*fn  v  ae  alao  significant  for  positive  work  environments. 
Art>us&l*ueckiinig  *;<;;idenr;y  is  a?*ot?.ated  with  an  individual's  response  to  stimuli,  i.e. 
anriety  tev&  Anxta?.*',     r  *\  a  action,  would  only  be  expected  to  be  detrimental  when  the 
level  of  j-rottMs  i&  much  bighe?  or  much  lower  than  the  learner's  preferred  arousal  level. 
When  looking  at  anxiety  le  ek  md  changes  in  those  levels,  it  is  probably  important  to 
consider  tbr*  learner's  e:  w*-xi  seeking  tendency.  Arousal- seeking  tendency  was, 
therefore,  included  as  l\  v>y:i&hh  in  this  study. 

Anxiety  Coping  Style 

Ho  w  iiihe  learnor  deils  with  anxiety  stimuli  are  also  important  factors  relating  to  the 
influence  of  instructional  interventions.  Tucker  (1986)  stated  that  regulatory  biases 
affect  the  activation  and  arousal  systems,  and  that  "the  most  adaptively  significant  is  the 
bks  towavd  internal  versus  external  determination  of  the  information  flow"  (p.  293). 
This  bias  ie  probably  associated  with  clearly  discernable  personality  traits  (Tucker  & 
WiliUJiisoti,  1984),  Some  learners,  when  at  high  anxiety  levels,  seek  external  stimuli  as 
their  preferred  source  of  information.  Other  learners  adapt  to  high  anxiety  levels  by 
focusing  rMmiion  inward,  thus  ignoring  outside  stimuli  and  information.  These 
internally  adaptive  individuals,  while  reducing  anxiety  levels,  would  probably  not 
undergo  cognitive  changes  in  relation  to  computer  skills  and  attitudes,  when 
experiencing  computer  anxiety  during  instruction. 

Miller  (1987)  has  suggested  such  a  personality  trait.  She  proposed  two  styles  for  coping 
during  stressful  events:  monitors  (externally  oriented,  information  seekers)  and 
blunters  (internally  oriented,  distractors).  For  example,  under  the  anxiety  of  an 
impending  surgical  procedure,  a  monitor  would  want  to  know  as  much  as  possible  about 
the  procedure  to  better  deal  with  the  stress.  A  blunter,  on  the  other  hand,  would  prefer  to  be 
told  as  liwtle  as  possible-ignorance  is  bliss.  Coping  style  was  defined  as  the  degree  to 
which  individuals  monitor  versus  blunt  stimuli  from  their  environment  when 
experiencing  anxiety.  Miller  asserts  that  the  degree  of  monitoring  and  blunting  can  be 
measured  using  the  Miller  Behavioral  Style  Scale  (MBSS). 

Several  studies  have  looked  at  coping  style  and  its  relationship  to  arousal  and  arousal- 
seeking  tendencies.  Sparks  and  Spirek  (1988)  reported  two  studies  aimed  at 
investigating  differences  between  high  monitoring  subjects  and  high  blunting  subjects. 
Results  showed  a  significant  main  effect  for  coping  style,  with  monitors  having  higher 
anxiety  levels  than  blunters.  Results  showed  that  f  ining  style  was  aignificumtly  related 
to  anxiety  levels  and  preferred  behaviors  toward  strt 

Hines  and  Mehrabian  (1979)  found  that  monitors  and  blunters  differed  significantly 
in  avoidance  of  unpleasant  settings.  Monitors  in  the  325  undergraduate  sample  showed 
more  avoidance  responses  to  unpleasant  stimuli  than  Aid  blunters. 

The  importance  of  copitig  s  tyle  to  instruction  and  anxiety  is  that  under  stressful 
computer  anxiety  in  an  irwiructicnal  setting,  blunter*  would  be  expected  to  prevent 
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assimilation  obstruction  and  potential  anxiety  reducing  interventions  by  blocking  or 
distracting  external  stimuli  (such  as  instructional  information).  Coping  style  can  have 
an  influence  on  changes  in  computer  anxiety  levels,  and  was,  therefore,  included  as  a 
variable  in  this  study. 

Locus  of  Control 

A  mtgor  cause  of  computer  anxiety  has  been  suggested  to  be  associated  with  a  feeling 
that  one  has  lost  control  of  outcomes  when  interacting  with  a  computer  (Bloom,  1985; 
Honeyman  &  White,  1987;  Meier,  1985).  The  focus  of  most  research  done  on  perceptions 
of  personal  control  has  involved  the  concept  of  locuo  of  control,  developed  out  of  social 
learning  theory  (Rotter,  1954;  Rottc.,  Chance,  &  Phares,  1972).  The  concept  of  locus  of 
control  emerged  due  to  systematic  differences  in  expectancies  following  reinforcement 
(Rotter,  1975).  Locus  of  control  (Rotter,  1966)  is  a  term  used  to  describe  a  generalized 
expectancy  for  internal  versus  external  control  of  reinforcements.  Belief  in  internal 
control  is  a  result  of  a  perception  that  events  are  contingent  upon  ones  behavior. 
Perceptions  of  external  control  are  views  that  reinforcements  are  the  result  of  such  factors 
as  luck,  chance,  or  fate.  Locus  of  control  does  not  refer  to  the  actual  extent  of  control  the 
individual  has  in  a  situation,  rather  it  refers  to  the  individuals  perception  of  control 
(Rotter,  1975). 

The  generalizability  of  locus  of  control  is  important.  Rotter  (1975)  emphasized  that 
specific  experiences  in  a  particular  situation,  not  only  determine  expectancies  for  that 
situation,  but  also  for  other  situations.  He  theorized  that  in  more  novel  or  ambiguous 
situations  (as  long  as  it  is  not  perceived  to  be  random)  the  learners  rely  on  their 
generalized  expectancy.  This  is  because  they  have  no  specific  expectancy  in  a  situation 
that  is  new  to  them.  Phares  (1976)  reported  that  personality  characteristics  have  been 
found  to  be  generally  associated  with  external  locus  of  control  individuals.  Among  these 
is  a  high  level  of  succorance,  a  need  for  assistance  under  conditions  of  distress. 
Externals  would,  therefore,  function  better  in  anxious  situations  when  help  or  assistance 
is  available. 

Because  computer  experiences  are  relatively  novel  for  beginners,  locus  of  control  has 
been  investigated  as  a  potential  correlate  to  computer  anxiety.  Studies  (Howard,  Murphy, 
&  Thomas,  1987;  Morrow,  Prell,  &  McElroy,  1986)  have  found  a  significant  relationship 
between  locus  of  control  and  computer  anxiety  in  college  students.  Lazarus  (1966) 
proposed  that  individuals  with  internal  locus  of  control  beliefs  are  better  equipped 
psychologically  to  handle  perceived  threats  and  are,  therefore,  less  likely  to  be  anxious  in 
a  threatening  situation.  Locus  of  control  was,  therefore,  investigated  as  a  variable  in  this 
study. 

Sex  Differences 

There  are  several  differences  between  male  students  and  female  students  that  would  be 
expected  to  impact  the  influence  of  instruction  on  computer  anxiety.  Wolleat,  Pedro, 
Becker,  and  Fennema  (1980)  administered  the  Mathematics  Attribution  Scale  (MAS)  to 
1224  secondary  students  enrolled  in  college  preparatory  mathematics  classes.  Females 
exhibited  significantly  more  of  the  learned  helplessness  (external  locus  for  success  and 
internal  locus  for  failure)  in  their  responses,  even  after  achievement  level  was  separated 
out.  These  types  of  differences,  although  potential  factors,  are  not  identifiable  with 
measures  of  locus  of  control. 

Mehrabian  (1977)  found  significant  sex  differences  in  coping  styles,  with  females 
more  likely  to  monitor  under  stress.  His  results  also  showed  that  these  monitoring 
coping  styles  increased  avoidance  responses  in  unpleasant  situations.  It  would  be 
expected  that  females  would,  therefore,  be  more  avoidant  of  computers  when  experiencing 
similar  levels  of  anxiety  as  males.  Males,  on  the  other  hand,  would  be  more  likely  to 
blunt  instruction  and  intervention  under  high  levels  of  computer  anxiety. 

Studies  (Cambre  &  Cook,  1987;  Raub,  1981;  Rohner  &  Simonson,  1981;  Rosen,  Sears,  & 
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w*ii  iQ*n  have  also  found  significant  correlations  of  computer  anxiety  levels  to  the 
*^!^^*SZ  h£  shown  females  to  possess  higher  levels  of  computer 


^"The'interaction  effects  of  the  sex  of  the  learner  with  other  variables «  ^  qjjto 
complex.  It  can,  however,  be  used  to  make  certain  predictions  of  behavior.  The  learner 
sex  was,  therefore,  included  as  a  variable  in  this  study. 

CXputeTanxTety  can  be  viewed  as  a  form  of  state  anxiety  associated  computer ^use 
(CambTe&Cook  1985;  Howard  &  Smith,  1986).  Computer  anxiety  was  defined  as  the 
f^J^M  ^  individuals  when  they  used  computers,  or  when  they 
considered  the  possibility  of  computer  utilization"  (Simonson,  et  al.,  1987  p.  238). 
£  A  1986;  Simonson,  et  al.,  1987)  have  used  state-anxiety  instruments 

to  validate  computer  anxiety  scales.  , 

Because  state  anxiety  is  transitory  in  nature,  its  levels  can  bechangedby 
interventions.  Thus,  anxiety  reducing  strategies  can  be  expected  to  ^™*J^*™ 
compter  anxiety.  Studies  (Cambre  &  Cook,  1987;  Howard,  Murphy,  &  Thomas  1987 
Zm981)  have,  in  fact,  shown  that  instruction  can  reduce  computer  anxiety  in  most 

leaWhile  research  about  students'  attitudes  toward  computers  was  performed  as  early  as 
19«RSatWs  Smkh  & Hansen,  1970),  the  study  of  computer  anxiety  as  a  construct  was 
i^S^SUing^  aid Talbott in  1973  (Cambre 

literature  on  computer  anxiety  involves  case  studies  (Howard  &  Kernan,  1989).  Other 
ISite^^rCook,  1987,  Howard  &  Smith,  1986;  Loyd  &  Gressard  1984;  R,ub, 
1981  Simonwn  at  al  1987)  have  looked  at  correlations  between  computer  anxiety  and 
toiW^A  Sd'ttie  overall  influence  of  computer  instruction  on  computer 
S  scores.  There  is  very  limited  data  regarding  the  effect  of  specific  ins  ructional 
Sods  on  computer  anxiety.  None  of  these  studies  had,  »  ^ '"JJ^^ 
relationship  of  specific  instructional  strategies  or  interventions i  to  computer ^ety 

Researchers  (Banks  &  Havice,  1989;  Bloom,  1985;  Wink  e  *  ** 
imoortance  of  using  instructional  strategies  to  reduce  anxiety-related  fears  toward 
computers  C  L  studies,  however,  have  actually  investigated  the  effect  of  instruction 
SS! Lls^omputer  anxiety.  Cambre  and  Cook  (1987)  investigated  factors  related 
^lam^LLtymd  overall  changes  in  computer  anxiety  scores  over  a  five-day  (10 

The  subjects  (N=865  pretreatment; N^70»ft 
who  signed  up  for  an  open  introductory  computer  workshop,  ranged  m  age  from  9  to  75 
vTars  Computer  anxiety  was  measured  by  response  to  one  embedded  item  I  am  afraid  to 
Si2 J  Precourse  computer  anxiety  was  significantly  related  to  sex,  but  not  age. 
Posteoum  computer  anxiety  v;as  significantly  related  to  age,  but  not  sex^  Due  to  complete 
anonymity  of  the  subjects,  individual  changes  in  computer  anxiety  could 
examined   The  overall  reported  computer  anxiety  levels  were,  however,  dramatically 

Honevman  and  White  (1987)  examined  the  effect  of  instruction  on  computer  anxiety 
basXon  the'vel  of  previous  computer  experience.  The  ^^ZT^e 
enrolled  in  an  introductory  computer  course  for  teachers  and  administrators  The 
treatment  involved  60  hours  of  instruction  on  the  use  of  application  software,  with 
apSty80%  of  the  instructional  time  working  on  the  computers  (two  persons  per 
computer)  The  STAI  was  used  to  measure  the  level  of  anxiety  with  computers  prior  to 
Smction,  at  the  midpoint  of  instruction,  and  the  penultimate  instructional  session. 
Sex  age  and  occupation  were  not  significantly  related  to  anxiety  scores.  Previous  level 
o Experience  was  only  related  to  beginning  state  anxiety. Overall; >te «»> 
and  last  half  anxiety  scores  were  significantly  improved  during  the  :ns^ction 

Howard  (1986)  then  used  a  pre-post  study  on  39  managers  enrolled  in  EMBA  eiaMte  * 
investigate  the  effect  of  a  microcomputer  training  session  on  computer  anxiety  *r,« 
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attitudes.  Subjects  were  randomly  assigned  into  the  treatment  and  control  groups.  The 
treatment  group  received  one  and  one-half  hours  of  instruction  and  hands-on  practice 
with  Lotus  1-2*3  and  use  of  DOS  commands,  while  the  control  group  received  a  lecture  on 
BASIC  programming  over  the  same  amount  of  time.  While  the  treatment  group  had 
marginal  reduction  of  computer  anxiety,  the  control  group  had  substantially  raised  level 
of  computer  anxiety.  Results  also  showed  no  significant  differences  in  treatment  effect 
between  the  high  anxiety  subjects  and  low  anxiety  subjects. 

Howard,  Murphy,  and  Thomas  (1987)  looked  at  the  effect  of  instructional  sequence  of 
BASIC  (an  unfriendly  programming  language)  and  VISICALC  (a  more  n/;er-friendly 
program)  on  computer  anxiety  and  the  relationship  between  computer  anxiety  and 
learner  characteristics.  One  treatment  group  was  taught  BASIC  before  ViSICALC,  while 
the  other  group  was  exposed  to  VISICALC  first.  The  44  subjects  wen*  3r>nrandomly 
assigned  into  treatment  groups.  While  computer  anxiety  scores  wwb  reduced  over  the 
five-week  introductory  computer  course  in  the  college  of  business,  there  were  no 
significant  differences  between  the  two  treatment  groups,  They  u-^ed  the  same 
instruments  as  Howard's  (1986)  previous  sv  iies,  Pretest,  computer  anxiety  was  found  to 
be  significantly  correlated  (p=.05)  to  locus  u  control,  math  aaxiety,  trait  anxiety, 
computer  knowledge,  computer  ev^rience,  and  class  rank. 

Because  computer  anxiety  is  a  perception  of  threat  n  #  fear  relating  to  computer 
interaction,  early  strategies  whirH  are  aimed  at  impjoving  confidence  would  be  expected 
to  act  as  a  positive  intervention  l-ehniqu?   Such  confidence-building  stratef  *es  in 
computer  instruction  would  be  expected  to  decrease  fear  of  failure  with  computers— a 
factor  in  computer  anxiety  (Johanson,  1985;  Rosen,  Seirs,  &  Weil,  1987).  Cattell  and 
Scheier  (1958)  concluded  trom  an  analysis  of  13  studies  investigating  over  800  variables 
relating  to  anxiety,  concluded  that  anxiety  does,  indeed,  appear  as  a  lack  of  confidence. 
Computer  confidence  in  this  study  trill  be  defined  as  the  degree  of  positive  expectancy  of 
ones  abilities  and  efficacy  felt  by  individuals  when  they  used  computers,  or  when  they 
considered  the  possibility  of  computer  utilization. 

M.  W.  Eysenck  (1979)  am?  Tobi&»  (1979)  stated  that  anxiety  interferes  with  attention  to 
task,  and  processing  a  id  encoding  of  information.  Based  upon  this  information- 
processing  mod?1,  they  recommend  that  instruction  for  anxious  learners  should  allow 
learners  to  repeat  content  and  reduce  reliance  on  memory.  Hands-on  practice  or 
experience  dunrg  the  ir  fraction  would  also  be  expected  to  help  maintain  the  learners' 
attention  to  the  instructional  task. 

Bloom  (iCi5)  ?tated  that  through  practice,  learners  who  experience  success  build  their 
confidence  level*,  Keller  (1983),  Keller  and  Kopp  (1987),  and  Keller  and  Suzuki  (1988),  as 
part  of  an  instructional  design  model  for  motivating  learners,  identified  learner 
r,  cv  ;tice  of  new  knowledge  in  a  supportive  atmosphere,  such  as  with  the  instructor 
"vailablft,  as  motivating  for  students.  Practice  as  a  component  of  leather  performance 
aft*r  demonstration  of  a  skill  is  an  element  of  Gagne's  (1977)  events  of  instruction. 
Pr^tice  is  "one  of  the  most  powerful  components  in  the  learning  process"  (Dick  &  Carey, 
1  :<*5,  p.  138). 

The  practice,  however,  may  be  most  helpful  if  it  immediately  follows  the 
demonstration  of  the  skill  during  instruction.  Widmer  anJ  Parker  (1983)  prescribed  the 
use  of  immediate  hands-on  practice  for  computer  learners  in  order  to  reduce  computer 
anxiety.  Ernest  and  Lightfoot  (1986)  and  Lewis  (1988)  also  suggested  that  computer 
anxiety  can  be  reduced  if  the  instructors  demonstration  of  computer  skills  be 
immediately  followed  by  learners'  hands-on  practice. 

Giving  learners  opportunities  for  hands-on  experience  and  practice  of  recently 
learned  computer  skills  during  instruction  allows  for  the  recommended  repetition  and 
improved  information  storage.  Furthermore,  as  the  learners  successfully  master 
computer  skills,  witl  the  help  of  these  strategies,  their  expectancy  for  success  increases. 
This  improvement  in  confidence  level  can  be  expected  to  reduce  feelings  of  fear-related 
anxiety. 
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The  literature  supports  the  use  of  instructional  interventions  to  reduce  anxiety  and 
fears  related  to  computers  during  learning.  Opportunities  to  practice  computer  skills, 
immediately  after  demonstration  and  during  the  instructional  process  is  believed  to  be 
an  important  strategy  for  the  reduction  of  computer  anxiety.  The  treatments  this  study, 
therefore,  investigated  were  the  use  of  hands-  on  computer  experience  and  practice  as  part 
of  the  instructional  process  and  absence  of  computer  practice  during  instruction.  Because 
the  learners  mastery  of  computer  skills  and  confidence  toward  computers  are  believed  to 
be  linked  to  computer  anxiety  reduction,  they  were  also  investigated  as  variables  in  this 
study. 

Methods 

This  study  used  multiple  repeated  measures  in  a  2  x  2  x  2  quasiexperimental  design 
with  in-class  hands-on  computer  experience  and  practice  during  instruction,  arousal- 
seeking  tendency,  coping  style,  sex,  computer  skill  mastery,  and  locus  of  control  as  the 
independent  variables.  Each  of  the  independent  variables  had  two  levels:  (a)  in-class 
hands-on  computer  experience  and  practice  during  instruction  (hands-on  computer 
experience  or  no  hands-on  computer  experience),  (b)  sex  (male  and  female),  (c)  arousal- 
seeking  tendency  (high  and  low  arousal  seeking),  (d)  coping  style  (high  and  !ow 
monitoring),  and  (e)  locus  of  control  (high  and  low  externality).  The  dependent 
variables  for  this  study,  pre  and  post  computer  anxiety  and  computer  confidence  scores, 
were  discrete  variables. 

Subjects 

The  subjects  were  120  students  (61  male,  58  female,  and  one  non-report)  enrolled  in  an 
undergraduate  level  introductory  computer  course  at  a  large  midwestern  university. 
This  course  is  a  computer  literacy  and  survey  course  with  computer  mastery  emphasis  in 
the  use  of  electronic  spreadsheets.  Although  this  is  a  required  course  in  the  College  of 
Business,  many  students  with  other  majors  enroll  in  this  course.   They  averaged  20.7 
years  of  age,  however  68%  were  either  19  or  20  years  of  age  at  the  beginning  of  the  study.  A 
majority  of  the  subjects  were,  therefore,  what  might  be  termed  traditional  freshmen  and 
sophomores. 

The  subjects  had  self-selected  into  the  specific  laboratory  and  lecture  sections. 
Random  assignment  of  subjects  into  treatment  groups  was  not  practical  in  this  type  of 
study  in  a  natural  setting.  However,  it  was  believed  that  assignment  into  sections,  to 
some  extent  performed  by  computer  because  of  the  high  demand  for  this  offering,  should 
provide  fairly  similar  treatment  groups. 

The  subjects  had  very  little  previous  experience  in  the  content  to  be  taught  during  the 
treatments.  At  the  beginning  of  the  study,  61%  of  the  subjects  reported  having  no  previous 
computer  spreadsheet  experience,  and  86%  reported  using  computers  with  spreadsheets  or 
databases  less  than  ten  times. 

Procedures 

The  two  graduate  assistants  who  volunteered  to  participate  each  had  one  of  their  two 
sections  randomly  assigned  to  one  of  the  two  treatment  levels,  the  other  section  was  then 
assigned  into  the  other  treatment.  Because  the  laboratory  sections  of  this  course  are 
traditionally  taught  in  a  lecture  style  with  demonstration  of  computer  skills  by  the 
instructors  without  the  students  at  computers,  the  sections  that  were  assigned  to  the  no 
hands-on  treatment  were  run  in  the  usual  manner,  without  hands-on  computer 
experience  and  practice  during  the  instruction.  The  section  assigned  to  in-class  hands- 
on  computer  experience  during  instruction,  met  in  a  different  location  for  the  eight  week 
treatment  period  of  the  study,  a  computer  lab  in  another  building  on  campus  that  was 
approximately  one-half  mile  away. 

The  subjects  were  given  the  pretests  during  the  first  lecture  session  of  the  second  week 
of  the  semester,  at  which  point  the  enrollments  had  become  relatively  stable.  Due  to  time 
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limits  and  constraints  by  the  coordinator  of  the  course,  the  pretest  instruments  were 
administered  in  the  lecture  class  at  a  point  when  all  of  the  treatment  laboratory  sections 
had  met  once.  After  the  eight-weeks  of  instructional  treatment,  the  subjects  were  then 
given  the  posttest  instruments  during  the  laboratory  class  (where  the  attendance  rate  was 
higher)  to  decrease  the  attrition  rate.  The  resulting  data  were  analyzed  based  upon  the 
research  questions. 

Three  subjects  in  each  treatment  section/class  (a  total  of  twelve  subjects)  were 
randomly  selecte  1  and  asked  to  complete  a  journal  of  their  thoughts  and  feelings  during 
the  period  of  the  treatment.  Seven  subjects  (five  males  and  two  females)  declined 
participation  and  other  randomly  selected  subjects  were  asked  to  volunteer  until  the 
twelve  volunteers  were  obtained. 

The  journals  were  collected  from  the  subjects  at  the  end  of  the  eight  week  treatment 
period.  Two  subjects,  one  from  each  of  the  hands-on  treatment  sections,  did  net  return 
their  journals  were  eliminated  from  the  analysis.  The  ten  returned  journals  were  coded 
and  analyzed  for  the  subjects'  cognitive  and  affective  perceptions  of  the  computer 
instruction  using  content  analysis  techniques  and  qualitative  matrix  analyses  to 
identify  trends  in  responses. 

Instructional  Treatments 

Instruction  was  primarily  similar  in  the  twe  treatments,  except  for  the  opportunity  for 
students  to  have  in-class  hands-on  computer  experience  and  practice  of  skills  during 
portions  of  the  instruction.  This  contrasted  with  the  other  treatment,  the  traditional 
instruction  for  these  classes,  which  consisted  of  the  instructors  demonstrating  the 
computer  skills  and  then  repeating  the  demonstrations  or  answering  students' 
questions.  In  both  treatments  the  instructors  demonstrated  the  computer  skills  using  a 
personal  computer,  a  liquid  crystal  display  panel,  and  overhead  projector.  The 
traditional  course  structure  only  gave  hands-on  experience  outside  of  class  while  the 
students  were  basically  on  their  own,  the  laboratory  room  aides  typically  only  helped  with 
hardware  problems.  Both  treatments  included  the  same  types  of  assignments,  requiring 
outside  hands-on  computer  experience  in  the  laboratory  room  or  some  other  computer 
environment.  In  each  group  students,  on  the  average,  waited  three  to  four  days  to  work  on 
the  computer  outside  of  class. 

The  content  covered  during  the  treatment  period  -  handling  of  floppy  discs,  basic  DOS 
commands,  and  spreadsheet  operations  -  was  the  same,  and  time  on  task  was 
approximately  equal  in  the  two  treatments.  Specifically,  the  instructional  treatment  was 
composed  of  eight  scheduled  50  minute  laboratory  sections,  one  per  week.  Students  were 
first  oriented  to  the  computer,  use  of  the  keyboard,  handling  of  diskettes,  and  basic  DOS 
commands.  Then  instruction  in  the  use  of  VP  Planner,  an  electronic  spreadsheet 
application  program,  was  given  during  the  last  six-weeks  of  the  treatment  period. 
Spreadsheet  skills  taught  during  the  treatment  ranged  from  relatively  simple  operations 
such  as  using  menus,  saving,  and  loading  files  to  more  complex  concepts  such  as  the  use 
of  mixed  cell  addresses,  formulas,  and  functions. 

On  the  average,  the  instructor  would  spend  the  first  15  minutes  of  the  class  to 
occasionally  take  attendance,  collect  assignments,  return  graded  work,  administer 
short  quizzes,  and  set  up  the  demonstration  computer,  projectot,  and  software.  Of  the 
remaining  280  minutes  of  instructional  time-on-task  in  the  lab  classes  during  the 
hands-on  treatment,  approximately  200  -  220  minutes  of  time  was  spent  by  the  students 
getting  hands-on  computer  experience  during  the  classroom  instruction.  Because  the 
completion  of  laboratory  assignments  required  computer  work  outside  of  class  in  the 
general  computer  laboratory  room,  all  students  in  both  treatments  working  on 
assignments  received  hands-on  computer  experience  to  some  extent  beyond  the 
manipulated  instructional  treatments. 

The  two  laboratory  assignments  during  the  treatment  period  involved  the  creation  of 
spreadsheets  using  formulas  or  functions  to  calculate  data,  a  budgetary  spreadsheet 
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(calculating  monthly  income  and  expenses  for  a  fictional  company)  and  a  financial 
spreadsheet  (calculating  monthly  payment  amounts  for  various  loan  amounts  at  a 
number  of  different  interest  rates).  Two  laboratory  quizzes  were  also  administered 
during  the  treatment  period  and  contributed  toward  the  subjects  laboratory  grade.  The 
first  quiz  covered  basic  information  about  floppy  diskettes  and  simple  spreadsheet  terms. 
The  second  quiz  was  over  spreadsheet  formulas,  operations,  and  more  advanced  terms. 

The  eight-week  treatment  period  was  used  in  the  design  of  this  study  for  reasons  of 
ethics.  This  design  allowed  learners  in  both  treatment  groups  to  have  approximately  half 
of  the  course  with  the  same  traditional  instructional  techniques.  Hands-on  computer 
experience  and  practice  during  instruction  was  not  expected  to  have  detrimental  effects 
on  learners.  If,  however,  the  hsnds-on  computer  experience  and  practice  during 
instruction  had  a  positive  effect,  as  expected,  on  the  subjects  over  the  course  of  the  study, 
then  this  design  diminished  any  disadvantage  the  other  treatment  group  or  those  not 
participating  in  the  study  (but  still  being  graded  with  those  in  the  practice  group)  may 
have  experienced  over  the  total  period  of  the  course.  Limited  availability  of  the  computer 
facility  did  not  allow  the  other  sections  to  have  later  access  to  the  hands-on  treatment  as 
an  alternative  to  the  design. 

Instructors 

Ore  faculty  member,  the  coordinator  of  the  course,  taught  the  two  large  lecture  classes 
for  all  twelve  sections  of  this  offering.  Six  graduate  assistants  each  taught  two  of  the 
laboratory  sections,  which  met  once  a  week  for  a  50  minute  session.  Two  of  the  six 
graduate  assistants  volunteered  to  participate  in  the  study.  Two  graduate  assistants 
expressed  an  interest  in  participating,  but  declined  due  to  scheduled  back-to-back 
classes.  It  was  not  practical  to  teach  consecutive  classes  ten  minutes  apart  in  two 
different  areas  of  the  campus.  The  other  two  instructors  declined  participation  in  the 
study  without  explanation  for  their  decision. 

Both  lab  instructors  participating  in  the  study  had  previous  experience  in  teaching  the 
laboratory  section  uf  the  course  a  id  were  Master's  Degree  level  Graduate  Assistants. 
They  both  stated  prior  to  the  study,  that  they  thought  the  hands-on  treatment  would  be  better 
fcr  the  students,  and  therefore  vere  happy  to  participate  in  the  study.  The  instructors  aid 
not  have  the  same  te^hing  style,  one  was  more  serious  but  was  better  organized  and  more 
helping  to  the  studer*_  .  Both  instructors  were  fairly  consistent  in  style  across  the 
treatments.  The  instructors  did  not  have  previous  experience  in  teaching  this  course  with 
students1  hands-on  computer  experience  during  the  instruction.  While  one  of  the 
instructors  adapted  quickly  to  the  new  teaching  environment,  the  other  had  some 
difficulty  keeping  the  entire  class  on  task  when  helping  individual  students  to  solve 
probhms  during  the  instruction.  After  the  third  week,  the  researcher  suggested  some 
teaching  strategies  to  that  instructor  in  order  to  improve  the  quality  of  hands-on  time  for 
the  subjects,  and  immediate  improvements  were  made. 

Instrumentation 

Subjects  were  given  a  packet  containing  a  battery  of  instruments  for  the  pretest  during 
th  eir  first  lecture  meeting  of  the  second  week  of  classes  in  the  spring  semester. 
Administration  of  th  e  pretest  to  tl  e  250  students  in  each  lecture  section  required 
approximately  35  minutes.  These  tests  have  been  selected  as  measures  of  the  variables 
investigated  in  this  study.  At  the  end  of  the  eight-week  study,  a  posttest  was  admii  istered 
to  the  subjects  in  the  treatment  laboratory  sections.  Administration  of  the  posttests  to  the 
approximately  40  subjects  in  each  laboratory  section  required  approximately  15  minutes. 

Computer  Anxjpty  Corrputer  anxiety,  defined  as  the  fear  or  apprehension  felt  by 
individu  Is  when  they  used  computers,  or  when  they  considered  the  possibility  of 
computer  utilization,  was  measured  with  the  Computer  Anxiety  Subscale  of  the  Computer 
Attitude  Scale  (CAS)  (Gressard  &  Loyd,  1984).  This  scale  contains  ten  items  and  uses  a 
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four-level  Likert-type  response  scheme  Loyd  and  Gressard  (1984)  report  this  subscale  of 
the  instrument  to  have  a  coefficient  alpha  reliability  of  .86.  The  instrument  was 
validated  by  judges  ratings  and  an  factor  analysis  of  the  ratings  of  155  subjects. 
Computer  anxiety  scores  may  range  frovn  10  to  40.  High  scores  represent  high  levels  of 
computer  anxiety  The  computer  anxiety  score  was  measured  by  summing  the  decoded 
responses  to  the  items  in  the  Computer  Anxiety  Subscale. 

Computer  Confidence.  Computer  confidence,  defined  as  the  degree  of  positive 
expectancy  of  ones  abilities  and  efficacy  felt  by  individuals  when  thev  used  computers,  or 
when  they  considered  the  possibility  of  computer  utilization,  was  measured  with  the 
Computer  Confidence  Subscale  (CCS)  of  the  CAS  (Loyd  &  Gressard,  1984a).  This  scale  is 
comprised  of  10  four-level  Likert-type  items.  The  CCS  has  a  .95  alpha  reliability 
coefficient.  Validity  was  shown  through  a  .73  correlation  to  computer  anxiety  scores  and 
a  factor  analysis  which  showed  loadings  for  the  computer  confidence  items  in  a  separate 
factor  from  the  computer  anxiety  and  liking  items.  The  computer  confidence  score  was 
measured  by  summing  the  decoded  responses  to  the  Computer  Confidence  Subscale.  CCS 
scores  may  range  from  10  to  40.  High  scores  represent  high  levels  of  computer 
confidence. 

Qoping  Style.  Coping  style,  defined  as  the  degree  to  which  individuals  monitor 
versus  blunt  stimuli  from  their  environment  when  experiencing  anxiety,  was  measured 
with  the  Miller  Behavioral  Style  Scale  (MBSS).  The  MBSS  consists  of  four  hypothetical 
stress  situations,  each  of  which  is  followed  by  four  monitoring  strategies  and  four 
blunting  strategies.  Subjects  selected  those  strategies  they  would  be  likely  to  use.  Miller 
(1987)  reports  the  MBSS  test-retest  reliability  over  a  4-month  period  for  this  scale,  r  =  .75, 
and  scores  have  been  found  to  be  unrelated  to  sex,  race,  and  age.  The  MBSS  was  validated 
experimentally,  with  blunting  scores  having  a  strong  negative  correlation  (-.79)  to 
observed  monitoring  behavior  and  the  alpha  coefficient  Tor  the  blunting  scale  =  .68 
(Miller,  1987).  Total  score  was  derived  by  subtracting  the  number  of  blunting  strategies 
selected  from  the  number  of  monitoring  strategies  chosen.  Scores  may  range  from  +lo  to 
-16,  and  positive  scores  represent  monitors  and  negative  scores  identify  blunters. 

Arousal-Seeking  Tendency.  The  arousal-seeking  tendency,  defined  as  the  extent  of 
the  level  of  arousal  that  an  individual  finds  most  comfortable,  was  assessed  with  the 
Measure  of  Arousal-Seeking  Tendency  (MAT)  (Mehrabian  &  Russell,  1974).  This  40- 
item  instrument  uses  a  nine  level  Likert-type  format.  The  MAT  has  a  four  to  seven  week 
test-retest  reliability  of  .88,  and  a  Kuder-Richardson  reliability  coefficient  =  .87 
(Mehrabian  &  Russell,  1974).  Scores  were  measured  by  summing  the  coded  responses  to 
all  forty  items.   Arousal-seeking  tendency  scores  on  this  scale  range  form  -160  to  +160. 
High  scores  on  this  scale  represented  high  arousal  seeking-tendency. 

Locus  of  Control.  Rotters  (1966)  29-item  Internal-External  Control  Scale  (IECS)  was 
used  to  measure  the  extent  to  which  each  subject  holds  generalized  external  control  beliefs 
versus  internal  control  beliefs.  Using  six  filler  items  to  disguise  the  purpose  of  the  test,  23 
internality-externality  items  are  used  to  determine  each  subjects  degree  of  externality. 
Scores  may  range  from  zero  to  23.  High  scores  on  this  scale  represented  a  more  external 
orientation.  These  generalized  beliefs  are  most  closely  associated  for  an  individual  in  a 
novel  situation.  The  IECS  has  reported  internal  reliability  estimates  from  .65  to  .79 
(Harrow  &  Ferrante,  1969).  Test-retest  reliabilities  over  a  six-week  period  have  been 
reported  as  .75  (Phares,  1976).  The  items  in  the  scale  wer*  validated  against  rating  on 
subjects  by  physicians  and  nurses  (Rotter,  1966). 

Computer  Experience.  The  amount  of  various  types  of  computer  experience  by  the 
subjects  before  and  after  treatment  was  assessed  by  questions  regarding  previous 
wordpro  cos  sing,  pr  rnramming,  spreadsheet,  database,  and  recreational  use.  Responses 
were  assigned  numerical  values  from  0  to  5  for  each  of  the  four  questions,  with  high 
scores  representing  more  computer  experience  in  each  of  the  four  categories. 

Computer  Skill  Ma  tery  Level.  The  degree  cf  computer  skill  mastery  was  measured 
by  using  the  average  c>  the  grade  5  obtained  for  computer  laboratory  class  during  the 
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eight-week  treatment  period.  The  four  grades  during  the  treatment  period  were  based  on 
two  in-class  quizzes  and  two  out-of-class  assignments  involving  applications  of 
spreadsheet  skills  taught  during  the  laboratory  class.  High  grades  represented  high 
computer  skill  mastery  levels  and  were  assigned  by  intervals  corresponding  to  ranges  of 
five  percentage  points. 

Data  Analysis 

Descriptive  statistics  for  this  study  include  means  for  each  treatment  group  in  the 
measures  of  computer  skill  mastery  level,  MBSS,  MAT,  IECS,  pre  and  post  Computer 
Anxiety,  pre  and  post  CCS,  scores.  The  number  of  male  and  female  subjects  and  subjects1 
ages  are  reported  for  the  total  sample  and  for  each  treatment  group.  Means  for  each  type  of 
computer  experience  for  both  treatment  groups  before  and  after  treatment  are  also 
reported.  The  mean  response  measures  of  the  perceived  interest,  relevance,  and 
satisfaction  levels  of  the  treatment  laboratory  classes,  attendance  rates,  and  time  lag 
after  class  until  working  on  computers  are  reported. 

In  order  to  address  the  research  questions,  analyses  used  two-way  analysis  of 
variance  (Time  X  Treatment)  on  one  repeated  factor  (Pre-  and  Post-  Computer  Anxiety; 
and  Pre-  and  Post-  Computer  Confidence),  three-way  analysis  of  variance  (Time  X 
Treatment  X  Locus  of  Control,  Sex,  Coping  Style,  and  Arousa,  Seeking  Tendency)  on  one 
repeated  factor  (Pre-  and  Post-  Computer  Anxiety),  and  partial  correlations  for  the 
combined  treatments  on  variables  (Computer  Skill  Mastery  Level  to  Computer  Anxiety; 
and  Computer  Skill  Mastery  Level  to  Computer  Confidence).  Alpha  for  these  statistical 
tests  was  set  at  0.05. 

The  subjects'  entries  into  the  journ^s  were  coded  using  content  analysis  techniques 
and  analyzed  using  qualitative  matrix  analysis  techniques  (Miles  &  Huberman,  1984). 
The  horizontal  categories  in  the  natrix  were  consecutive  two-  week  time  periods  during 
the  study.  The  vertical  categories  were  identified  through  a  content  analysis  of  the 
subjects1  journal  entries,  including  both  positive  and  negative  perceptions  of  the  subjects. 

Data 

One  hundred  forty-seven  subjects  participated  in  the  study.  Twenty-seven  subjects 
were  eliminated  from  data  analysis,  due  to  attrition  and  excessive  missing  or  out-of- 
range  responses.  The  in-class  hands-on  computer  experience  during  instruction 
treatment  group  contained  51  subjects  and  the  no  hands-on  computer  experience  during 
instruction  treatment  contained  69  subjects.  Sixty-one  percent  of  the  total  subjects,  and 
61%  in  each  treatment  group  reported  they  had  never  used  a  computer  spreadsheet  or 
database  before  the  study. 

Table  1 

Comparisons  of  Mean  Scores  for  Hands-On  and  Non-Hands-On  Treatment  Groups  on 
Computer  Anxiety,  Computer  Confidence,  Locus  of  Control,  Arousal-Seeking  Tendency, 
and  Coping  Style  Measures 


Treatment 

Computer 

Computer 

Locus 

Arousal- 

Coping 

Group 

Anxiety 

Confidence 

of 

Seeking 

Style 

Control 

Tendency 

Pre  Post 

Pre  Post 

Hands-on 

17.9  17.5 

30.5  31.6 

10.3 

-19.7 

1.79 

No  Hands-on 

20.3  19.3 

29.5  30.0 

10.3 

-26.0 

3.11 
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The  distributions  by  sex  in  the  hands-on  and  no  hands-on  treatment  groups  were  26 
males  and  25  females,  and  35  males  and  33  females,  respectively.  The  mean  age  of 
subjects  in  each  group  was  21  years.  The  mean  pre-  and  post-  anxiety,  pre-  and  post- 
confidence,  locus  of  control,  arousal-seeking  tendency,  and  coping  style  scores  for  the 
hands-on  treatment  group  and  the  other  treatment  group  are  summarized  in  Table  1. 
Pretest  computer  anxiety  and  coping  styles  were  significantly  different  (p  <  .05)  for  the 
two  treatment  groups. 

The  two  groups  had  similar  computer  experience  prior  to  the  study  (See  Table  2).  The 
only  significant  (a  =  .05)  differences  in  reported  previous  experience  between  the 
treatment  groups  was  in  the  category  of  computer  programming  with  the  hands-on 
subjects  averaging  approximately  six  more  programming  experiences  than  the  other 
treatment  subjects.  The  largest  gain  in  computer  experience  for  both  treatment  groups 
was  in  the  category  of  database  and  spreadsheet  use,  because  the  lab  instruction  primarily 
covered  computer  spreadsheet  applications.  The  subjects  in  each  group  waited  an  average 
of  three  to  four  days  to  work  on  computers  after  of  class. 

Table  2 

Comparisons  of  Means  for  Hands-On  and  Non-Hands-On  Treatment  Groups  on  Pre- 
and  Post  Measures  of  Computer  Experience  for  Wordprocessing,  Spreadsheet/Database, 
Programming,  and  Recreation 


Treatment  Database/ 

Group  Wordprocessing       Spreadsheet        Programming  Recreation 


Pre 

Post 

Pre 

Post 

Pre 

Post 

Hands-on 
No  Hands-on 

1.69 
1.70 

1.92 
2.07 

0.73 
0.70 

1.51 
1.42 

1.73 
1.12 

2.06 
1.39 

2.82 
2.83 

3.22 
3.15 

NOTE.  1  =  none;  2  =  one  to  10  times;  3  =  11  to  20  times;  etc. 

Analyses  of  variance  for  the  eight  research  hypotheses  showed  no  significant 
differences  between  the  treatment  groups,  or  the  interactions,  on  the  repeated  measures  of 
computer  anxiety  and  computer  confidence.  Analyses  of  the  journal  entries,  however, 
suggest  some  perceptual  or  experiential  differences  between  the  groups. 

Journals 

Categories  of  the  subjects'  perceptions  were  determined  on  the  basis  of  the  content  of  the 
entries  in  the  ten  journals.  Journal  entries  were  then  coded  by  category,  treatment,  sex, 
and  time  period.  Frequencies  of  coded  entries  were  placed  into  a  matrix  (See  Table  9, 
next  page)  in  order  to  analyze  changes  over  time  and  to  identify  trends  in  the  entries  by 
treatment  group. 

Some  trends  from  the  matrix  analysis  include  the  different  distribution  of  comments 
between  the  two  groups  regarding  either  dislike  for  the  laboratory  instruction  or  dislike 
for  the  learning  environment.  Two  entries  (or  .50  entries  per  subject)  from  the  journals 
of  the  hands-on  during  instruction  treatment  subji  cts  and  14  entries  (or  2.33  entries  per 
subject)  from  the  other  treatment  subjects  expressi  d  a  dislike  for  lab  instruction  or 
environment  Many  more  subjects  in  the  treatments  not  having  hands-on  experience 
during  instruction  were  negative  about  the  laboratory  learning  experience. 

A  second  unbalanced  distribution  was  also  identified  from  entries  describing  a 
liking  for  the  laboratory  instruction  and  learning  environment.  Six  comments  (or  1.50 
entries  per  subject)  by  the  hands-on  treatment  and  two  (or  .33  entries  per  subject)  by  the 
other  treatment  were  about  liking  the  laboratory  sessions.  Hands-on  treatment  subjects 
made  positive  statements  about  the  laboratory  learning  experience  at  a  higher  rate. 
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Table  9 

Matrix  of  the  Ratio  of  Coded  Comments  in  Journals  to  Number  of  Subjects  in  the  Hands- 
On  and  Non-Hands-On  Treatment  Groups  by  Two  Week  Time  Period 

Comment  First  Second  Third  Fourth 

Category  Two  Weeks      Two  Weeks      Two  Weeks      Two  Weeks 


H* 

H 

N 

H 

N 

11 

N 

Anxiety 

.75 

.33 

.25 

.00 

.00 

.00 

.00 

.17 

Negative  Feelings 

.50 

.00 

.25 

.17 

.25 

.50 

.00 

.17 

Dislike  Lab  Instructn. 

.00 

.33 

.25 

.33 

.00 

.50 

.25 

.33 

Dislike  Lab  Envirnmt. 

.00 

.67 

.00 

.00 

.00 

.00 

.00 

.17 

Confidence 

.75 

.50 

.25 

.00 

.25 

.33 

.25 

.00 

Positive  Feelings 

.25 

.25 

.00 

.17 

.00 

.00 

.25 

.67 

Like  Lab  Instruction 

.25 

.17 

.25 

.17 

.00 

.17 

.25 

.00 

Like  Lab  Envirnmt  . 

.25 

.00 

.00 

.00 

.25 

.00 

.25 

.00 

Internalization 

.00 

.00 

.00 

.00 

.00 

.17 

.75 

.17 

NOTE.  Values  shown  are  the  ratio  of  responses  to  the  total  number  of  subjects  in  group 

*    Hands-on  During  Instruction  Treatment  Group  n  =  4 

**  No  Hands-on  During  Instruction  Treatment  Group  n  =  6 

Entries  specifically  about  the  laboratory  environment  also  showed  differences 
between  the  treatment  groups.  All  three  positive  comments  about  the  laboratory 
environment  were  made  by  subjects  in  the  hano^on  treatment  group.  All  five  negative 
comments  about  the  laboratory  environment,  w  the  other  hand,  were  by  subjects  in  the 
other  treatment  group. 

Another  pattern  appeared  as  a  change  in  the  ratio  of  entries  over  time.  In  the  category 
of  negative  feelings,  the  ratio  of  the  number  of  entries  per  subject  by  the  hands-on 
treatment  subjects  to  the  number  of  entries  per  subject  by  the  other  treatment  subjects 
changed  from  4.50:1.00  over  the  first  four  weeks  to  1.00:2.33  over  the  last  four  weeks  of  the 
study.  Entries  were  also  analyzed  by  sex  of  the  subject.  While  female  subjects  did  report 
more  thoughts  and  feelings  overall  than  males,  there  were  no  discernable  patterns  of 
either  positive  or  negative  entries  favoring  either  sex. 

Discussion 

The  nonsignificant  results  of  the  analyses  of  variance  may  be  due  to  the  she;  t 
treatment  time.  Honeyman  and  White  (1987)  only  found  significantly  improved  state 
anxiety  scores  for  subjects  after  approximately  24  hours  of  hands-on  computer 
instruction.  Howard,  Murphy  and  Thomas  (1987)  found  no  differential  effects  by 
treatment  on  computer  anxiety  after  approximately  12  hours  of  hands-on  computer 
instruction.  These  studies  suggest  that  more  hands-on  computer  experience  during 
instruction,  than  the  approximately  three  and  one-half  hours  of  treatment  in  this  study, 
may  be  necessary  to  show  significant  differences  between  treatments. 

The  design  of  the  study  limited  the  treatment  time  to  one-half  of  the  laboratory 
instruction  in  the  course  as  a  compromise  between  the  desire  to  have  the  maximum 
amount  of  treatment  and  the  ethical  considerations  of  all  of  the  students  enrolled  in  the 
course.  The  researcher  did  not  want  to  put  any  students,  whether  participating  or  not 
participating  in  the  study,  at  a  large  disadvantage  of  any  type  over  the  entire  period  of  the 
course.  Grades  of  some  students  on  some  of  the  assignments  might  have  been  influenced 
by  the  experiences  related  to  the  treatments.  Limits  on  the  availability  of  the  computer 
facility  used  in  the  study  did  not  allow  for  alternative  designs  which  might  ethically 
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allow  a  longer  treatment  period,  such  as  letting  all  sections  use  the  facility  the  second 
half  of  the  semester. 

The  low  nonsignificant  relationship  of  computer  skill  mastery  level  with  computer 
confidence  and  anxiety  is  suggested  within  some  of  the  anxiety  literature.  Eysenck 
(1984)  explains  that  the  performance  levels  of  high  anxiety  and  low  anxiety  subjects  does 
not  differ  by  a  large  degree.  He  su  marizes  the  research  that  suggests  highly  anxious 
learners  typically  compensate  for  decreased  cognitive  abilities  (due  to  anxiety)  by 
increasing  their  effort  on  the  learning  task. 

Analysis  of  variance  calculations  showed  computer  anxiety  for  the  total  sample  to 
vary  significantly  by  subjects1  externality.  This  fact  supports  the  computer  anxiety 
literature,  that  locus  of  control  is  more  directly  related  to  computer  anxiety  than  is 
achievement.  The  perception  of  control  over  ones  success  or  failure  with  computers  is 
related  to  the  degree  of  anxiety  one  feels  when  interacting  with  computers  (Bloom,  1985; 
Honeyman  &  White,  1987;  Meier,  1985). 

Changes  in  computer  confidence  significantly  interacted  with  time  for  the  total 
sample,  while  changes  in  computer  anxiety  did  not  significantly  interact  with  time, 
This  suggests  that  the  effect  of  both  treatments  during  the  three  and  one-half  hours  of  on 
task  instruction  over  eight  weeks  is  sufficient  to  produce  significant  changes  in  the 
confidence  of  learners.  This  also  suggests  that  significant  changes  in  confidence 
toward  computers  may  take  place  more  easily  than  changes  in  computer  anxiety,  or  that 
the  computer  confidence  subscale  may  be  more  sensitive  than  the  computer  anxiety 
subscale  to  the  changes  occurring  in  the  subjects  during  the  study. 

While,  statistical  analyses  showed  no  significant  differences  on  the  measures  in  the 
eight  research  hypotheses,  analyses  of  the  journal  entries  of  the  subjects  in  the  two 
treatments  did  suggest  some  differences.  The  hands-on  computer  experience  during 
instruction  treatment  subjects  showed  more  positive  and  fewer  negative  journal  entries 
than  the  other  treatment.  The  hands-on  treatment  groups  also  reported  a  decreased 
number  of  negative  feelings,  while  the  other  treatment  groups  recorded  an  increased 
number  of  negative  feelings  from  the  first  half  to  the  last  half  of  the  study.  These  patterns 
suggest  that  some  differences  by  treatment  groups,  not  significantly  measurable  by  the 
instruments,  may  have  occurred  in  the  thoughts  and  feelings  of  the  subjects  during  the 
period  of  the  study. 

When  the  hands-on  treatment  subjects  were  asked  on  the  posttest  "How  did  your 
having  an  opportunity  to  work  at  the  computer  during  the  computer  lab  classes  affect  your 
learning?",  53%  of  those  subjects  responded  that  "it  helped  my  learning  a  lot,"  and  86%  of 
those  subjects  reported  that  it  helped  to  some  extent.  When  asked  a  similar  question  "How 
did  your  having  an  oppor-tunity  to  work  at  the  computer  during  the  computer  lab  classes 
affect  your  confidence?"  32%  of  those  subjects  responded  that  it  helped  a  lot,  and  82% 
reported  that  it  helped  to  some  extent 

In  spite  of  the  implied  changes  in  the  affect  of  the  subjects,  these  results  suggest  that 
more  than  the  eight  hours  of  instruction  and  three  and  one-half  hours  of  hands-on 
computer  practice  during  instruction  may  be  necessary  to  significantly  change  learners1 
computer  anxiety  scores  over  non-hands-on  treatments.  Especially  when  working  on 
relatively  advanced  computer  skills, such  as  spreadsheet  applications,  in  an  introductory 
computer  class  with  a  minimum  of  laboratory  experience,  more  time  interacting  with 
computers  may  be  needed  to  make  a  significant  impact  on  the  computer  anxiety  levels  of 
students. 

Attendance  of  the  laboratory  sections  differed  significantly  by  treatment  group,  only 
41%  of  the  hands-on  treatment  group  reported  attending  all  eight  laboratory  classes 
during  the  study,  while  74%  of  the  subjects  in  the  other  treatment  group  reported  attending 
all  eight  laboratory  sections.  The  fact  that  both  hands-on  treatment  groups  met  on  Friday 
afternoon,  while  the  other  treatments  met  on  Monday  and  Wednesday  afternoon,  may 
account  for  the  attendance  differences.  Th  overall  effect  may  have  been  minimal,  only 
11.8%  and  7.2%  of  the  hands-on  and  other  treatment  group  subjects,  respectively,  missed 
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more  than  one  class.  A  cause  of  this  difference  in  attendance  rates  may  have  been  the 
different  laboratory  class  location,  about  one-half  mile  from  the  Business  College 
building,  for  the  hands-on  treatment  group. 

Another  problem  involved  a  potential  leveling  effect.  On  the  pretest  measure  of 
computer  anxiety,  10  subjects  (8.3%  of  the  sample)  scored  a  ten—the  lowest  possible  score- 
s-reflecting a  lack  of  measurable  computer  anxiety.  These  subjects  would  not  be  able  to 
show  improvement  to  their  posttest  scores.  Eight  other  subjects  (6.7%  of  the  sample)  scored 
an  eleven  and  had  little  chance  of  significantly  improving  their  scores.  A  majority  of 
these  (ten  subjects)  were  in  the  hands-on  treatment  groups  (20%  of  that  sample)  compared 
to  eight  in  the  non-hands-on  (12%),  were  in  this  range  of  extremely  low  computer  anxiety 
scores.  These  differences  may  have  impacted  on  the  results. 

Additionally,  the  instructors  had  not  previously  taught  this  class  in  a  computer 
laboratory.  One  of  the  instructors  was  able  to  actfust  the  instruction  in  the  new 
environment  to  keep  the  subjects  on  task  when  giving  individual  help.  The  other 
instructor,  however,  had  some  difficultly.  Only  after  three  weeks  of  treatment  did  this 
instructor  become  comfortable  with  teaching  in  the  new  environmer^.  Several  of  the 
students  were  observed  to  be  visibly  distressed  by  the  lack  of  attention  for  up  to  five 
minutes  at  a  time.  This  confounding  condition  may  have  diminished  the  effects  of  the 
hands-on  treatment  for  some  students  in  that  treatment  class. 

Summary 

Analyses  of  sex,  arousal  seeking  tendency,  coping  style,  locus  of  control,  and 
computer  skill  mastery  on  changes  in  computer  anxiety  and  confidence  showed  no 
significant  differences  by  treatment  Additional  qualitative  and  quantitative  data, 
however,  suggested  that  the  treatments  may  have  had  different  affective  effects  on  the 
some  perceptions  of  the  subjects.  These  results  suggest  that  for  short  periods  of  time 
hands-on  vs.  non-hands-on  instruction  may  not  make  very  much  difference  in  the 
anxiety,  confidence,  and  performance  of  young  relatively  motivated  learners.  However 
if  other  affective  concerns,  such  as  attitudes,  are  important  factors,  then  hands-on 
computer  instruction  is  preferred. 

A  need  for  further  research  is  suggested  using  similar  treatments  to  this  study,  but 
with  more  sections  involved,  over  a  longer  time  period,  with  a  delayed  post-test  the 
following  semester.  Such  a  study  would  better  examine  the  effects  of  the  treatment  in  a 
natural  setting  over  the  typical  time  period  of  an  introductory  course,  as  well  as  the  long 
range  effects  of  the  instruction,  while  reducing  some  of  the  confounding  variables  and 
limitations  of  this  study. 
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THE  PSYCHOLOGY  OF  HYPERMEDIA: 
A  CONCEPTUAL  FRAMEWORK  FOR  R&D 


Introduction 

"More  new  information  has  been  produced  in  the  last  30  years  than  in  the  previous 
5,000.  About  1,000  books  are  published  internationally  every  day  and  the  total  of  all 
printed  knowledge  doubles  every  eight  years," 

—  Peter  Large 

"We  are  like  a  thirsty  person  who  has  been  condemned  to  use  a  thimble  to  drink 
from  a  fire  hydrant. 

—  Richard  Saul  Wurman 

Think  about  a  close  friend.  Perhaps  she  is  an  accountant.  The  concept  of  accountant  is 
associated  with  many  other  concepts  and  ideas  —  taxes,  green  visors,  a  love  of  numbers,  etc.  Your 
friend  has  many  attributes  associated  with  her  (e.g., she  is  slender,  she  likes  horror  movies,  she 
dresses  elegantly,  etc.)  only  one  of  which  is  that  she  is  an  accountant 

Now  suppose  a  colleague  complains  about  having  to  prepare  this  year's  tax  return.  Your 
colleague's  grousing  triggers  your  thoughts  about  the  concept  of  "accountant"  which  are,  in  turn, 
associated  with  your  friend.  Thinking  about  your  friend  now  may  remind  you  thauyou  are 
scheduled  to  see  a  movie  together  on  Friday,  which  may  cause  you  to  think  of  one  of  your  favorite 
horror  movies. 

The  mind  seems  to  operate  through  associations.  How  else,  then,  could  one  go  from  a 
thought  of  accounting  to  thinking  about  horror  stories?  Thought^  often  trigger  other  thoughts. 
Indeed,  mental  concentration  is  nothing  more  than  the  effort  exerted  to  keep  the  mind  from 
associating  too  much— to  keep  it  focused  on  one  thing  at  a  time.  Without  the  mind's  penchant  for 
associations,  how  could  classical  or  operant  conditioning  be  so  powerful?  Why  would  the  creativity 
technique  of  mindmapping  have  become  so  popular?  Why  would  hearing  an  old  song  send  many 
into  an  ecstasy  of  nostalgia?  And  why  else  would  educators  assert  that  the  best  way  to  make  new 
material  meaningful  is  to  relate  it  to  knowledge  already  in  the  brain?  However  the  brain  works,  it 
is  anything  but  linear. 

Consider  how  one  might  classify  a  Dalmatian  dog.  As  a  dog?  A  pet?  A  spotted  animal?  A 
mammal?  A  fireman's  companion?  Clearly,  the  world  resists  classification  into  single,  linear 
hierarchies;  a  Dalmatian  can  be  associated  with  any  number  of  other  things.  Now  try  to  imagine 
the  best  wav  to  classify  a  mouse.  As  a  member  of  a  ^roup  of  animals  called  rodents?  As  a  strange 
pet  to  have?  As  an  undesirable  house  companion?  Mice,  like  Dalmatians  and  everything  else  in  the 
world,  can  fall  under  any  number  of  classification  schemes.  Indeed,  the  Dalmatian  and  the 
mouse,  both  mammals,  can  also  be  classified  as  popular  animated  Disney  characters. 

The  meaning  we  ascribe  to  a  concept  depends  entirely  on  which  classification  scheme  or 
set  of  associations  is  accessed.  This  access  depends  on  the  situation  (i.e.  context).  Indeed,  to  the 
age-old  question,  "What  is  the  meaning  of  life?",  the  answer  is  probably,  "It  depends."  It  depends 
on  how  we  look  at  things,  when  we  look  at  them,  and  even  to  what  things  we  choose  to  attend.  In 
other  words,  it  depends  upon  how  we  construct  our  personal  reality — how  we  perceive  the  world 
and  even  what  we  choose  to  include  in  our  view  of  it. 

If  both  the  brain  and  the  world  around  us  consist  of  networks  of  highly  interconnected 
ideas,  concepts,  processes  and  events,  then  linear  educational  media  such  as  books  and  videotape 
are  sorely  inadeauate  to  deal  with  this  ever-exr  nding  universe  of  information-  highly 
interconnected,  dynamic  information.  A  new  way  of  accessing  and  processing  this  information  is 
needei. 

This  "new"  idea  was  actually  proposed  almost  fifty  years  ago.  The  man  who  proposed  this 
new  scheme  knew  much  about  the  associative  nature  of  both  the  brain  and  the  world  and  also 
knew  about  how  meek  the  tools  of  his  time  were  for  storing  accessing  and  using  a  steadily 
accumulating  mass  of  information.  The  year  was  1945  and  the  man  was  Vannevar  Bush,  science 
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advisor  to  Franklin  Roosevelt.  Then,  as  now,  scientific  knowledge  was  accumulating  at  a  rate  far 
exceeding  anyone's  ability  to  make  effective  use  of  it.  Bush  believed  that  to  enhance  tne  acquisition 
of  knowledge  and  to  assist  in  viewing  information  in  new  and  useful  ways,  one  might  wish  to 
follow  various  trails,  perhaps  to  wander  off  on  a  tangent  and  later  return  to  the  mainstream.  For 
this  endeavor,  paper  was  clearly  inadequate.  Bush's  answer  to  the  paper  dilemma  was  the 
memex.  In  his  own  words, 

A  memex  is  a  device  in  which  an  individual  stcres  all  his  books,  records,  and 
communications,  and  which  is  mechanized  so  that  it  may  be  consulted  with 
exceeding  speed  and  flexibility.  It  is  an  enlarged  intimate  supplement  to  his 
memory,  (pp.  106-107). 

Bush  later  states, 

It  affords  an  immediate  step,  however,  to  associative  indexing,  the  basic  idea  of 
which  is  a  provision  whereby  any  item  may  be  caused  at  will  to  select  immediately 
and  automr tically  another.  This  is  the  essential  feature  of  the  memex.  The  process 
of  tying  two  items  together  is  the  important  thing,  (pp.  107). 

Although  the  memex  was  never  built,  Bush  set  forth  a  vision  that  took  almost  half  a  century  to 
materialize.  The  memex  was  the  conceptual  parent  of  what  we  now  call  hypertext. 


Hypertext/Hypermedia 


Imagine  a  new  accessibility  and  excitement  that  can  unseat  the  video  narcosis  that 
now  sits  on  our  land  like  a  fog.  Imagine  a  new  libertarian  literature  with 
alternative  explanations  so  anyone  can  choose  the  pathway  or  approach  that  best 
suits  him  or  her;  with  ideas  accessible  and  interesting  to  everyone,  so  that  a  new 
richness  and  freedom  can  come  to  the  human  experience;  imagine  a  rebirth  of 
literacy.  (Nelson,  1987;  p.  1/4) 

Despite  its  proclamation  as  a  revolutionary  information  and  learning  tool,  as  a  weapon 
believed  to  be  a  bastion  of  democracy  and  freedom,  hypertext  has  always  seemed  like  a  solution  in 
search  of  a  problem — or  perhaps  a  solution  whose  problems  might  outweigh  its  benefits.  Yet  it  has 
an  intuitive  appeal  that  persists.  It  seems  to  be  one  of  those  things  that  we  Know  is  right  because  it 
just  feels  right.  It  is  this  intuitive  appeal,  recently  proposed  cognitive  learning  theories,  and 
dizzying  progress  in  technology  that  have  synergistically  fueled  the  hypertext  mania. 

most  discussions  about  hypertext  tend  to  expound  the  virtues  ofhypertext  and  then  go  on  to 
explain  the  fortuitous  coincidence  of  how  this  new  technology  happens  to  mirror  the  functioning  of 
the  brain — a  solution  looking  for  a  problem  to  solve.  The  approach  taken  in  this  paper  is  the 
inverse  of  such  an  approach.  We  will  start  with  an  examination  of  theories  and  studies  regarding 
how  people  learn,  access  and  use  information  and  ideas,  and  only  then  move  to  consider  the  new 
technology  that  may  miiror  the  way  our  brains  work. 

Brief  Glossary  of  Terms 

What  exactly  is  hypermedia  anyway?  There  is  often  some  confusion  regarding  exactly  what 
hypermedia  is  and  what  it  is  not.  Five  terms — interactive  video,  interactive  multimedia, 
multimedia,  hypertext  and  hypermedia — are  often  used  interchangeably  yet  represent  distinct 
ideas. 

Interactive  video  (IV)  combines  the  "interactive  capability  of  the  compute?*  with  the  unique 
properties  of  video  presentations**  (Chen,  1990;  p.  6).  The  typical  interactive  video  arrangement 
includes  a  videodisc  player  connected  to  a  personal  computer,  with  the  computer  controlling  the 
presentation  of  the  video  in  response  to  interaction  with  the  user.  IV  refers  specifically  to  the 
computer-videodisc  player  combination.  IV  is  a  subset  of  interactive  multimedia  (see  Evans,  1986). 

Interactive  multimedia  refers  to  any  computer-based  configuration  in  which  some 
combination  of  video,  computer-generated  graphics,  sound,  animation,  and  voice  is  used  (see 
Ambron  and  Hooper,  1988,  1990;  Stefanac  and  Weiman,  1990).  These  multimedia  elements  can 
either  come  from  external  sources  such  as  videodisc  players,  VCRs  or  audio  equipment,  or  they 
can  be  generated  internally  such  as  when  video,  sound  or  graphics  are  stored  digitally  on  a  hard 
disk  or  optical  disk.  What  gives  interactive  multimedia  its  interactivity  is  the  computer.  Without 
the  computer,  the  result  is  simply  multimedia. 
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Of  all  the  five  terms,  multimedia  is  the  most  difficult  to  define.  The  term  has  been  used  to 
refer  to  everything  from  slide  shows  to  extravaganzas  complete  with  multiple  monitors, 
animation,  video,  sound  and  text.  It  would  be  easy  to  remember  that  multimedia  stands  for 
multiple  media  except  that  the  term  media  can  mean  many  things.  "Media"  can  include  slides, 
audiotapes,  videotapes,  videoconferencing,  animation,  films,  music,  voice,  paper,  or  even  someone 
shouting  through  a  megaphone.  Media  can  be  instructional  or  not;  it  can  be  interactive  or  not;  and 
it  can  be  computer-based  or  not.  While  the  focus  of  interactive  video,  interactive  multimedia,  and 
multimedia  is  on  the  technology,  the  focus  of  hypermedia  is  on  method—a  method  that  so  happens 
is  most  suited  to  the  computer. 

Understanding  hypermedia  depends  on  an  understanding  of  its  related  term, 
hypertext.  Quite  different  from  the  continuous  flow  of  text  inherent  in  traditional 
presentations  of  information,  hypertext  breaks  up  the  flow  into  modules.  "By  modularizing 
information,  users  have  the  option  of  selecting  the  next  module,  which  may  include  an 
elaboration  or  example  of  the  present  idea  or  an  entirely  new  idea  to  be  compared  with  the 
previous  one."  (Jonassen,  1989;  p.  7). 

Hypertext  is  often  described  as  a  computer-based  method  of  nonsequential  reading 
and  writing— a  technique  with  which  chunks,  or  nodes,  of  information  can  be  arranged 
and  rearranged  according  to  an  individual's  needs,  previous  knowledge,  curiosities,  etc. 
(Conklin,  1987;  Jonassen,  1989;  van  Dam,  1988;  Begoray,  1990;  Hall  and  Papadopoulos, 
1990).  Hypertext  not  only  presents  information  as  a  book  does,  it  also  represents 
information.  As  discussed  earlier  in  this  paper,  meaning  is  derived  from  the  arrangement 
of  information.  The  implication  here  is  two-fold:  1)  Meaning  is  not  static,  but  rather, 
dynamic— changing  as  the  arrangement  of  nodes  changes,  and  2)  Meaning  resides  in  the 
relationships  between  the  nodes,  not  in  the  nodes  themselves.  This  insight  is  intriguing 
and  has  exciting  implications  for  learning.  According  to  Kearsley  (1988),  hypertext  should 
improve  learning  by  focusing  attention  on  the  relationships  between  ideas  rather  than  on 
isolated  facts.  Associations  provided  by  the  links  in  hypertext  should  facilitate  retrieval, 
concept  formation,  and  comprehension. 

Hypermedia,  a  term  often  used  interchangeably  with  hypertext,  is  simply  multimedia 
hypertext  (Nielsen,  1990).  That  is,  hypermedia  extends  hypertext  to  include  the  full  range  of 
information  forms:  sound,  graphics,  animation,  video,  music,  as  well  as  text.  As  hyp  rmedia  is 
the  more  inclusive  of  the  t\,o  terms,  this  paper  will  use  the  term  hypermedia  as  a  catch-all  term 
refernngto  both  text-based  and  multiple  media-based  systems. 

v  £  ™mainder  of  this  PaPer  will  explore  the  insights  that  psychology  can  offer  in  shedding 

light  on  why  hypermedia  may  work  and  thus  in  suggesting  fruitful  areas  for  future  research.  It  is 
in  this  spirit  that  we  should  explore  some  of  the  psychological  underpinnings  of  hypermedia. 


Some  Psychological  Underpinnings  of  Hypertext 


Humans  are  perceived  by  cognitive  theorists  as  being  processors  of  information,  acquiring 
knowledge  and  skills  by  way  of  interactive  internal  processes.  These  internal  processes  are 
orchestrated  by  a  set  of  control  processes  which  are  personal,  reflective  of  both  innate  abilities  and 
individual  experience. 

Theories  proposed  by  cognitive  psychologists  as  well  as  strategies  employed  by  cognitively 
oriented  educators  and  instructional  designers  exhibit  a  basic  cohesion  in  that  humans  are  viewed 
as  processing  information  by  actively  attending  to  stimuli,  accessing  existing  knowledge  to  relate 
to  new  information,  realigning  the  structure  of  that  existing  knowledge  to  accommodate  new 
information,  and  encoding  restructured  knowledge  into  memory  (Jonassen,  1985).  The  meaning 
that  the  individual  ascribes  to  information  is  viewed  as  idiosyncratic,  actively  constructed  from  a 
base  of  existing  knowledge  (Winn,  1988).  Our  active  participation  in  the  construction  of  this  new 
knowledge  is  viewed  as  a  critical  component  for  accessing  and  retrieving  prior  knowledge  in  the 
interpretation  of  new  information  (Weinstein  1978).  Thus,  as  we  explore  some  of  the  ideas  of  what 
we  know  about  how  we  learn  and  apply  knowledge,  it  becomes  obvious  that  activity  as  well  as 
interactivity  are  integral  components  of  both  theory  and  its  application  in  the  technology  of 
hypermedia. 

Cognitive  Flexibility  Theory 

Many  instructional  systems  fail,  claiio  Rand  Spiro  and  colleagues,  because  they  do  not  take 
into  consideration  the  dl-structuredness  of  many  complex  domains  (Spiro  and  Jehng,  1990;  Spiro, 
Feltovich,  Jacobson,  and  Coulson,  1991).  Cognitive  flexibility  theory  is  an  attempt  to  explain  how 
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the  brain  makes  sense  of  ill-structured  domains  (i.e.  domains  that  are  complex  and  in  which  there 
is  little  regularity  from  one  case  to  another). 

Essentially,  the  theory  states  that  cognitive  flexibility  is  needed  in  order  to  construct  an 
ensemble  of  conceptual  and  case  representations  necessary  to  understand  a  particular  probiem- 
solving  situation.  The  idea  is  that  we  cannot  be  said  to  have  a  full  understanding  of  a  domain 
unless  we  ha  e  the  opportunity  to  see  different  case  representations. 

According  to  Spiro  et  al.,  advanced  knowledge  requires  an  approach  to  learning  that  is 
quite  different  from  that  of  introductory  learning.  The  fundamental  problem  running  throughout 
many  learning  efforts  is  "oversimplification"  Oversimplification  occurs  in  different  forms.  For 
example,  addxtivity  bias  occurs  when  parts  of  complex  entities  are  studied  and  it  is  assumed  that 
these  parts  could  be  then  re-integrated  as  a  whole  while  retaining  their  original  complex 
characteristics.  Another  example  is  discreteness  bias,  where  continuous  processes  are  divided  up 
into  discrete  steps.  Yet  another  form  of  oversimplification  includes  the  compartmentalization  bias, 
in  which  highly  interrelated,  interdependent  concepts  are  considered  in  isolation  from  one 
another  without  consideration  of  how  they  interact. 

Cognitive  flexibility  theory  is  an  answer  to  the  necessity  of  mastering  complex  subjects 
without  falling  prey  to  oversimplification  in  any  or  all  of  its  forms.  Basically,  the  theory  posits  that 
in  order  to  learn  complex  material,  the  learner  has  to  see  that  sanr  material  at  different  times,  in 
different  situations,  for  different  purposes  and  from  different  conceptual  perspectives  (Spiro, 
F  ,ovich,  Jacubson,  and  Coulson,  1991).  Complete  schemata  are  constructed  to  the  extent  that 
learners  ara  exposed  to  multiple  knowledge  representations  and  multiple  interconnectedness  of 
the  complex  material  to  be  learned. 

Given  the  tenets  of  cognitive  flexibility  theory  and  what  we  know  about  the  unique 
characteristics  of  hypermedia,  it  is  easy  to  see  the  promising  fit  between  the  technology  of 
hypermedia  and  the  manner  in  which  we  allegedly  best  learn  complex  subject  material.  To  make 
the  potential  fit  of  hypermedia  to  cognitive  flexibility  theory  even  stronger,  recall  that  hypermedia 
is  nothing  more  than  a  tool  to  link  together  nodes  (chunks  of  information,  or  "conceptual 
elements  )  together  in  meaningful  ways.  This  capability  matches  precisely  the  requirements  of 
cognitive  flexibility  theory — the  ability  to  see  different  aspects  of  a  complex  subject  area  from 
different  perspectives  in  different  contexts. 

Hypermedia,  therefore,  has  properties  that  are  consistent  with  cognitive  flexibility  theory. 
For  example,  conceptual  elements  in  the  hypermedia  environment  can  be  re-arranged  so  that 
several  conceptual  elements  can  be  viewed  together.  For  another  purpose,  or  in  a  different 
situation,  conceptual  elements  could  be  arranged  differently.  Recall  tnat  it  is  the  arrangement  and 
relationships  among  nodes  that  give  meaning  to  information.  By  freely  rearranging  nodes 
according  to  specified  features/properties  depending  on  different  needs,  different  contexts  and 
different  problems,  learners  have  a  powerful  tool  with  which  to  learn  and  understand  complex 
material  in  all  its  forms,  in  different  situations,  and  in  its  entirety, 

Even  though  hypermedia  has  been  accused  of  being  a  solution  in  search  of  a  problem,  it  is 
exciting  to  contemplate  the  idea  that  hypermedia  may  be  just  the  opposite,  a  solution  to  the  very 
tricky  problem  of  how  to  effectively  and  efficiently  teach  complex,  multifaceted  subjects  with  many 
interconnected  elements,  such  as  medicine,  literature,  economics,  etc.  Given  the  almost  made-to- 
measure  fit  of  hypermedia  to  cognitive  flexibility  theory,  it  is  important  that  we  make  it  our  top 
priority  to  take  a  closer  look  at  now  it  fits  and,  perhaps  more  importantly,  why  it  fits.  Further 
research  is  required  to  clarify  exactly  how  the  brain  deals  with  complex  subject  matter  and  how 
hypermedia  systems  can  be  shaped  to  facilitate  optimal  learning. 

Information  Processing 

Information  processing  is  proposed  as  an  interactive  system.  An  individual  uses  the 
physical  senses  to  make  sense  out  of  the  stimuli  of  the  environment.  This  sensory  recognition  is 
interrelated  with  the  individual's  long-term  memory,  experiences,  abilities,  and  expectations. 
Information  is  first  received  and  recognized  and  is  then  either  encoded  anr1  stored  for  later 
retrieval  or  is  lost.  The  stages  of  information  processing—  recognition,  short-term  (working) 
memory  storage,  and  long-term  memory  storage —  are  dependent  on  internal  processes,  what  we 
remember,  therefore,  is  a  result  of  interactions  within  this  system  of  active  internal  processes. 

The  3ystem  processes  information  by  first  recognizing  patterns.  If  information  is  first 
recognized,  ^t  can  then  take  on  meaning.  To  extend  our  discussion  of  the  concept  of  the  Dalmatian 
dog,  for  example,  we  would  first  process  the  information  presented  by  recognizing  the  pattern  of 
the  letters  DOG,  The  combined  alphabet  letters  D.  0.  and  G,  however,  are  not  just  recognized  as 
individual  letters     terms  of  how  they  are  shaped;  they  represent  a  host  of  associations  which 
have  already  been    mentioned.  Whether  we  are  ble  to  recognize  this  word  and  jrive  it  meaning 
each  time  we  see  the  letters  DOG  depends  on  whether  the  information  about  DuG  is  stored  in  long- 
term  memory  and  can  be  retrieved  into  short-term  memory  (working  memory)  when  called  upon. 


063 


According  to  information  processing  theory,  retention  of  information  is  attributed  to  basic 
processes— attention,  selective  perception,  rehearsal,  encoding,  and  retrieval— all  of  which  occur 
throughout  the  three  im«rmation  processing  stages.  Currant  theorists  put  more  emphasis  on 
these  processes  and  how  they  are  affected  rather  than  the  stages  in  which  they  occur.  To  increase 
the  probability  that  information  is  acquired,  stored,  and  retrieved,  we  must  somehow  affect 
attention,  selective  perception,  rehearsal,  encoding,  and  retrieval.  As  processing  occurs, 
information  and  skills  are  recoded  to  form  stronger  links  to  nodes  and  more  pathways  to  stored 
information.  According  to  theorists,  information  must  be  adequately  encoded,  stored,  and 
retrieved  to  be  remembered.  Thus,  any  retention  aid  should  encourage  extra  processing  of 
information  to  increase  links  to  stored  skills.  According  to  Tulving  and  Thomson  (1973),  the  degree 
of  accessibility  of  informution  stored  in  long-term  memory  is  dependent  on  the  strength  of  the  cues 
encoded  during  acquisition.  Strong  cues,  such  as  those  cues  which  may  be  generated  in 
hypermedia,  are  more  likely  to  be  present  at  recall. 

Case-Based  Reasoning 

Case-based  reasoning  is  an  exciting  educational  paradigm  in  which  learning  depends  on 
memory  of  specific  experiences,  or  "cases"  (Riesbeck  and  Schank,  1989).  Medical  and  law  schools 
have  long  utilized  case-based  instruction  techniques  in  their  curricula.  Cases  provide  the  means 
with  which  students  can  learn  why,  when,  and  how  knowledge  is  applicable.  In  order  to  avoid  the 
inert  knowledge  problem"  (see  Whitehead,  1929;  Bransford,  Sherwood,  Vye,  &  Rieoer,  1986), 
learners  must  have  the  opportunity  to  organize  new  knowledge  appropriately  and  know  how  and 
in  what  situations  to  apply  the  knowledge.  The  important  wint  here  is  that  instruction  is  more 
effective  when  anchored  in  meaningful  problem-solving  environments  similar  to  those  in  which 
students  will  find  themselves  when  solving  actual  real-world  problems. 

If  the  usefulness  of  information  depends  on  an  ability  to  recall  it  when  appropriate,  then  it 
is  important  that  cases  be  presented  in  such  a  way  as  to  provide  contextual  cues  to  properly  index 
information  in  memory.  Hypermedia  may  facilitate  this  indexing  by  presenting  information  along 
with  appropriate  contextual  cues.  Ferguson  et  al.  (1991)  have  created  a  hypermedia  case-based 
learning  environment  that  helps  organize  information  the  same  way  in  which  the  information 
will  have  to  be  organized  when  applied  to  some  task. 

Generative  Learning 

Cognitive  principles  suggest  that  learning  is  an  active,  constructive  process  in  which 
learners  generate  meaning  for  infoimation  by  accessing  and  applying  existing  knowledge. 
anfu      e  learnin?  theory  incorporates  these  principles  of  cognitive  psychology  (Wittrock,  1974a, 
1974b,  1979).  The  theory  maintains  that  meaning  for  material  is  generated  by  activating  and 
altering  existing  knowledge  structures  to  interpret  new  information  and  encode  it  effectively  for 
future  retrieval  and  use.  In  Wittrock's  (1974b)  model,  meaningfu,  learning  is  a  rational  t.nd 
transferable  process: 

...  the  generative  model  predicts  that  learning  is  a  function  of  the  abstract  and 
distinctive,  concrete  associations  which  the  learner  generates  between  his  prior 
experience,  as  it  is  stored  in  long-term  memory,  and  the  stimuli.  Learning  with 
understanding,  which  is  defined  by  long  term  memory  plus  transfer  to 
conceptually  related  probLn  s,  is  a  process  of  generating  semantic  and  distinctive 
idiosyncratic  associations  between  stimuli  and  stored  information,  (p.  89) 

Comprehension,  according  to  the  generative  model,  requires  the  proactive  transfer  of 
existing  knowledge  to  new  material.  Wittrock  (1990)  maintains,  for  example,  that  generative 
reading  focuses  on  constructive  or  generative  processes  usually  considered  characteristic  of  good 
written  composition.  In  other  words,  reading,  like  good  writing,  involves  generative  cognitive 
processes  that  create  meaning  by  building  relations  among  the  parts  of  the  text  to  be  learned  as 
well  as  between  the  text  and  what  the  learner  knows,  believes,  and  experiences.  Learning 
activities  which  require  the  learner  to  relate  new  information  to  an  existing  knowledge  structure 
depend  on  complex  cognitive  transformations  and  elaborations  that  are  individual,  personal,  and 
contextual  in  nature.  Generative  learning  activities,  then,  can  be  thought  of  as  strategies  which 
foster  not  only  learning  but  learning-to-learn  (Brown,  Campione,  &  Day,  1981).  In  these  learning 
activities,  information  is  transformed  and  elaborated  into  a  more  individual  form  for  the  learner 
thereby  making  the  information  more  memorable  as  well  as  more  comprehensible 
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The  nature  of  traditional  mudia  such  as  paper  and  video  forces  a  linearity  onto  its  content. 
Authors  must  arrange  their  work  hierarchically  so  that  one  idea  flows  to  the  next  and  where  ideas 
are  subsumed  under  other  ideas.  As  we  have  discussed  earlier,  however,  the  mind  is  anything  but 
a  linear  thinking  machine.  Ideas  are  not  arranged  as  on  a  scroll  with  one  neatly  leading  to  the 
next.  In  fact,  it  appears  that  the  mind  consists  of  endlessly  intertwined  webs  of  interrelated  ideas, 
emotions  and  skills. 

Quiliian  (1968)  developed  the  idea  of  'Wive  structural  i  etworks  (ASN)  in  an  effort  to  capture 
and  depict  the  ostensibly  network-like  nature  of  the  brain.  These  networks  consist  of  nodes  (chunks 
of  information)  and  labeled  links  (relationships)  between  nodes.  Allegedly,  everything  someone 
mighi  know  could  be  encoded  into  the  active  structural  networks  of  the  brain.  Furthermore, 
networks  serve  as  structures  wUhin  which  new  information  is  integrated  with  knowledge  already 
possessed  by  the  learner.  The  implication,  then,  is  that  learning  involves  the  acquisition  of  new 
chunks  of  information  and  their  connection  to  related  chunks  of  information  both  new  and  old. 

It  is  easy  to  see  how  active  structural  networks  function  by  speaking  in  terms  of  schema 
theory.  Scherrata  consist  of  interconnected  sets  of  ideas  which  are  farther  linked  to  other 
schemata.  Meaningful  learning  takes  place  when  new  ideas  are  linked  to  other  new  idoas  and 
when  all  aru  somehow  assimilated  into  existing  networks  of  schemata.  While  it  might  seem  that 
the  connections  between  nodes  and  between  schemata  are  fixed  except  when  new  learning  is 
taking  place,  the  fact  is  quite  the  opposite. 'Relationships  within  and  between  schemata  are  defined 
in  terms  of  context.  The  implication  is  that  we  remember  and  process  ideas  differently  depending 
on  the  context. 

It  follows  that  it  does  not  make  sense  to  speak  of  an  Absolute  Truth  about  anything  in  our 
world,  then,  since  the  meaning  of  anything  depends  on  how  we  perceive  it  in  relation  to  otner 
things.  Recall  our  discussion  of  the  Dalmatian  dog.  It  is  not  adequately  stating  the  Truth  or  the 
final  word  about  Dalmatian  dogs  to  say  that  they  are  members  of  a  genus,  the  canines,  within  the 
class  called  mammals.  We  are  no  closer  to  absolute  Truth  if  we  add  that  they  are  members  of  a 
class  of  animals  that  are  spotted.  The  Dalmatian  dog  is,  in  fact,  a  member  of  an  extremely  rich 
and  diverse  collection  of  interconnected  ideas.  It  is  a  type  of  carnivore  which  is  a  family  of 
mammals,  but  it  is  also  merely  a  dog.  Dogs  are  associated  with  other  concepts  such  as 
companionship,  love,  and  with  loud  noises.  They  might  also  be  associated  with  the  thought  of  a 
good  friend  who  owns  four  of  them.  The  Dalmatian  dog  might  also  be  connected  to  memories  of 
childhood  when  the  people  next  door  had  one.  We  could  go  on,  ad  infinitum,  adding  more  and 
more  to  our  Dalmatian  dog  schemata,  but  we  could  only  approach  the  Truth,  never  arrive  at  it.  As 
this  demonstration  suggests,  meaning  is  constructed  by  each  individual  and  is  highly  dependent 
on  context.  In  learning,  accessing  and  applying  knowledge,  only  a  small  subset  of  the  Truth  is 
ever  relevant.  Indeed,  this  position  is  at  the  foundation  of  the  incre  singly  popular  educational 
movement,  constructivism,  Suchman  (198  J)  argues  that  there  is  no  ultimate  reality.  Instead, 
reality  is  relative  and  is  the  outcome  of  a  constructive  process.  Ii  is  one's  experience  with  an  idea 
and  the  context  of  which  the  idea  is  a  part  that  act  together  to  construct  a  meaning  of  that  idea 
(Brown,  Collins,  and  Duguid,  1989). 

All  of  this  brings  us  back  to  the  limitations  of  paper,  video,  and  other  linear  systems.  If 
learning  involves  the  manipulation  of  networks  of  information  and  if  the  relationships  binding 
these  networks  change  depending  on  the  context,  the  inadequacy  of  linear  systems  becomes  clear. 
Within  hypermedia,  information  can  be  rearranged,  analyzed,  shifted,  and  molded  to  suit  the 
needs  of  each  individual  and  the  context  in  which  the  material  is  learned  and/or  applied. 
Hypermedia  is  a  technology  which  can  assist  its  users  in  constructing  their  schemata  according  to 
context  and  to  individual  needs  and  characteristics.  The  exact  ways  in  which  hypermedia  can  be 
made  to  facilitate  schemata  processing,  however,  requires  a  good  deal  more  empirical  research. 

Dual  Coding  Theory 

The  central  theme  underlying  semantic  network  theory  is  that  information  is  not  simply 
filed  away  under  a  single  heading,  but  rather  is  integrated  in  somo  way  into  a  highly 
interconnected  network  of  data  and  the  relationships  between  those  data.  Dual  coding  theory  (DCT) 
goes  further  in  that  it  postulates  that  mental  representations  of  ideas  and  events  are  stored  in 
distinct  verbal  and  nonverbal  symbolic  modes  and  that  these  representations  retain  the  pattern  of 
sensorimotor  activation  present  during  encoding  (Clark  and  Paivio,  1991;  Paivio,  198P\ 

One  of  the  structural  assumptions  underlying  DCT  is  that  connections  are  formed  both 
within  verbal  or  nonverbal  representations  (associative  connections)  and  between  the  two  forms  of 
representation  (referential  connections).  For  example,  an  associative  connection  would  be  one  in 
which  thinking  about  computers  might  remind  you  of  related  terms  such  as  monitor,  printer,  and 
perhaps  even  frustration.  A  referential  connection  would  be  present  if  the  word  "mouse*  conjured 
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up  memories  of  images  of  mice,  the  squeaking  sounds  they  make  and  a  sensatin"  of  anxiety.  Thus, 
both  inter  and  mfra-associative  structures  exist  between  and  within  the  two  stems. 

Mental  representations  are  activated  by  related  nodes  in  the  network  *^ed  nodes. 

Thoughts  can  activate  associated  nodes  and  then  spiead,  resulting  in  a  complex  4   .vern  of 
spreading  activation  among  nodes  and  links  in  the  network.  Returning  to  our  example  of  the 
dog,the  sight  of  a  Dalmatian  dog  might  trigger  the  image  of  your  neignbor  who  owns  a  Dalmatian. 
The  image  of  your  neighbor  might,  in  turn,  activate  the  thought  of  "dentist"  as  that  is  your 
neighbors  occupation,  which  might  arouse  a  sense  of  uneasiness  as  you  remember  your  last  visit 
to  tne  dentist. 

This  last  example  points  out  an  important  assumption  of  DCT  specifically  as  well  as  active 
structural  networks  generally.  As  activation  of  nodes  and  associations  spread  out,  they  tend  to 
weaken.  It  is  easy  to  see  how  a  thought  of  a  Dalmatian  could  activate  the  thought  of  the  dog's 
owner.  It  becomes  increasingly  ridiculous  to  speak  of  strong  associations  between  verbal  and 
nonverbal  mental  representations  because,  ultimately,  everything  is  related!  DCT,  then,  provides  a 
plausibb  explanation  of  how  the  brain  deals  with  information,  in  both  verbal  and  nonverbal  forms, 
which  are  more  or  less  related  to  one  another. 

Another  interesting  aspect  of  DCT  relevant  to  our  discussion  of  hypermedia  is  that  it 
emphasizes  the  role  of  past  experience  in  the  development  of  mental  representations.  Although 
two  individuals  may  have  the  same  experiences,  their  mental  representations  of  these  experiences 
will  be  different  in  significant  ways,  a  function  of  their  past  experiences.  Insights  derived  from 
DCT  are  consistent  with  the  position  taken  by  constructivists  who  maintain  that  there  is  no  Truth 
out  there,  but  rather  that  each  individual's  mental  representation  of  reality  is  a  constructed 
version  of  reality,  and  each  person's  construction  is  different  from  all  others. 

DCT  makes  it  easy  to  see  how  beneficial  hypermedia  can  be.  First,  one  of  the  implications  of 
DCT  is  that  the  more  sensory  modes  in  which  mental  representations  are  stored,  the  more  likely  it 
is  that  they  will  be  remembered.  Hypermedia  makes  it  easy  to  access  related  information  in  a  wide 
variety  of  forms.  A  picture  of  a  human  heart  can  be  accompanied  by  a  motion  video  segment 
depicting  blood  flow  through  the  ventricles,  detailed  drawings,  textual  explanations  of  anatomy 
and  processes,  and  heartbeat  sounds.  All  of  this  information  could  be  immediately  available  at  the 
click  of  an  on-screen  button.  The  quick  and  easy  retrieval  of  related  information  in  its  different 
forms  helps  to  build  stronger  relationships  between  these  pieces  of  information  than  if  one  had  to 
stop  reading  a  textbook,  retrieve  a  videotape,  then  an  audiotape,  then  a  carousel  of  slides,  etc. 

Another  implication  of  DCT  is  that  the  mental  representations  constructed  depend  on 
context.  If  different  people  construct  different  representations  of  r  eality  in  order  to  conform  to  the 
context  and  their  experiences,  then  it  follows  that  any  tool  that  helps  people  to  personalize  the 
information  structure  would  be  beneficial. 

The  idea  of  spreading  activation  suggests  that  it  might  be  useful  to  assign  values  to  links 
emanating  from  each  node.  The  values  assigned  would  represent  the  strength  of  the  conceptual 
relationship  between  nodes.  These  values  would  indicate  to  users  which  of  several  links  are  the 
strongest,  and  would  thus  help  users  decide  which  links  to  traverse.  Of  course,  there  is  no  reason 
why  users  could  not  be  provided  with  the  means  of  assigning  values  to  links  themselves.  The 
implications  of  active  structural  networks  for  hypermeaia  are  many  and  exciting.  Much  research 
remains  to  be  done  in  this  area. 

Dual  coding  theory  has  much  to  recommend  hypermedia.  It  is  a  comprehensive  theory  of 
learning  and  suggests  many  implications  for  education.  Its  main  idea  is  that  all  knowledge  is 
constructed  in  the  brain  in  many  different  forms  and  is  different  from  person  to  person.  All  of  this 
points  up  the  need  for  learning  and  information  acquisition  tools  that  facilitate  associative 
processing  of  information  in  a  variety  of  forms,  which  are,  in  fact,  integral  to  hypermedia. 

Bridging  the  Gap 

What  is  hypertext?  Hypertext  is  freedom!  Freedom  from  the  burdens  of  traditional 
computing  applications.  Freedom  from  fear  of  failure.  Every  hypertext  user 
succeeds  in  getting  somewhere  and  getting  something.  (Shneiderman  and 
Kearsley,  1989;  p.  2) 

All  of  this  is  manifest  destiny.  There  is  no  point  in  arguing  it;  either  you  see  it  or 
you  don't.  Many  readers  will  choke  and  fling  down  the  book,  only  to  have  the 
thought  gnaw  gradually  until  they  see  its  inevitability  (Nelson,  1987;  p.  0/12) 

An  attempt  should  be  made  to  bridge  the  gap  between  what  we  know  about  how  we  learn 
and  apply  knowledge  and  a  new  technology  that  promises  to  augment  the  brain's  functioning  in  a 
way  nevsr  befor ,  possible.  Hypertext  and  hypermedia  have  been  trumpeted  as  revolutionary 
devices  to  help  us  deal  with  the  geometrically  increasing  pool  of  information  in  the  world.  Among 
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the  many  advantages  hypermedia,  claims  include  that  it  increases  learner  involvement  and 
learner  control,  it  focuses  on  relationships  rather  than  isolated  bits  of  data,  it  permits  levels  of 
explanations  and  it  is  easy  to  do  annotations  of  all  kinds. 

All  of  these  benefits  are  kudos  to  the  technology  of  hypermedia  itself.  The  technology 
permits  information  to  be  revealed  as  necessary  (levels  of  explanation)  in  whatever 
order/arrangement  is  necessary  (focus  on  relationships)  and  among  other  things,  provides  a  nice 
method  for  uncovering  side  information  or  adding  one's  own  side  notes  (annotation).  The  real 
intrigue  of  hypermedia,  however,  derives  from  its  unique  ensemble  of  characteristics  which 
tantalizingly  corroborate  evidence  regarding  the  way  our  brains  work. 

The  idea  of  a  machine  based  on  associative  indexing  is  not  new  (Bush,  1945),  but  the 
technology  to  materialize  the  vision  is  only  now  becoming  available,  with  exciting  new  capabilities 
just  over  the  horizon.  While  we  explore  the  possibilities,  however,  we  must  not  forget  that  the 
technology  must  work  for  humans,  not  the  converse. 

The  philosophy  underlying  this  paper  is  that  research  into  the  instructional  efficacy  of 
hypermedia  should  begin  with  a  knowledge  and  appreciation  of  relevant  psychological 
phenomena.  This  not  to  say  that  we  should  retreat,  learn  about  how  we  learn  and  then  return  with 
an  armload  of  instructional  strategies  to  be  applied  to  the  new  technology.  Ross  and  Morrison 
(1989)  put  it  well, 

Aside  from  being  difficult  to  accomplish,  separating  media  from  methods  may  not 
always  be  desirable  in  instructional  technology  research.  Although  media  do  not 
directly  affect  learning,  they  serve  as  influential  moderating  variables  through 
their  effects  on  learner  attributions  and  their  differential  properties  for  conveying 
instructional  strategies.(pp.  29-30). 

Knowledge  of  new  technologies  offer  us  new  opportunities  for  understanding  how  we  learn 
as  they  provide  new  capabilities.  In  turn,  knowledge  of  how  we  learn  feeds  back  to  guide  the 
development  of  th^  new  technologies,  creating  a  wonderful  cycle  of  progress.  Hypermedia  has 
unique  characteristics  that  point  us  to  relevant  areas  of  learning  research.  Hopefully,  an 
examination  of  the  psychology  and  learning  phenomena  will,  in  turn,  inform  the  research  and 
development  of  hypermedia,  which  will  then  feed  back  to  learning  and  psychology  research, 
creating  a  most  fecund  feedback  loop.  Perhaps  the  conceptual  father  of  hypermedia  said  it  best 
when  he  wrote: 

There  is  a  new  profession  of  trail  blazers,  those  who  find  delight  in  the  task  of 
establishing  useful  trails  through  the  enormous  mass  of  the  common  record. 
...Thus  science  may  implement  the  ways  in  which  man  produces,  stores,  and 
consults  the  record  of  the  race.  (Bush,  1945;  p.  108) 

We  are  in  virgin  territory.  What  is  more,  we  are  exploring  this  unknown  territory  in  the 
dark.  Every  bit  of  knowledge  we  have  about  how  we  learn,  process,  and  apply  knowledge  is  like  a 
spotlight  beaming  down  on  a  small  part  of  the  foreign  landscape.  The  more  psychological 
knowledge  we  consider,  the  more  spots  will  become  illuminated,  making  more  and  more  of  the 
terrain  visible  and  understood.  But  the  spotlights  are  not  erratic.  They  correspond  to  specific 
paths,  much  the  same  as  streetlights  light  up  not  the  whole  city,  but  specific  avenues  throughout 
the  city.  Our  paths  lie  within  the  psychology  of  learning.  When  aimed  down  on  the  unknown 
territory  of  the  new  technology,  spotlights  could  illuminate  the  right  paths  that  we  must  take  to  use 
the  technology  to  enhance  our  ability  to  learn,  process,  and  apply  knowledge.  This  paper  is  an 
attempt  to  fire  up  as  many  of  these  spotlights  as  p  jssible  in  order  to  light  the  way. 
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The  premise  is  advanced  that  a  major  weakness  of  the  everyday  generic 
instructional  systems  design  model  stems  from  a  too  modest  traditional  conception 
of  the  purpose  and  potential  of  formative  evaluation.  In  the  typical  ISD  model 
formative  evaluation  is  shown  not  at  all  or  as  a  single,  product  evaluation  step.  Yet, 
in  practice  formative  process  evaluation  is  also  widely  accepted  as  desirable  and  is 
more  pervasive  than  the  models  would  indicate  because  it  is  done  informally.  A 
broader  theory  of  product-plus-process  formative  evaluation  for  instructional 
design  is  offered.  The  critical  factors  of  this  descriptive  theory  focus  upon  the 
desirability  and  viability  of  introducing  formative  evaluation  techniques  at  the  very 
beginning  of  the  ISD  process  rather  than  at  the  end.  A  revised  purpose  of  formative 
evaluation  is  theorized  which  envisions  an  ISD  system  that  provides  process 
feedback  as  well  as  product  feedback,  thus  enabling  the  system  to  become  truly  self 
correcting.  In  the  theoretical  str  icture  posited,  the  output  of  any  observable  or 
definable  step  in  the  ISD  process  is  a  deliverable.  Furthermore,  the  case  is  stated 
that  not  only  are  all  of  these  deliverables  things  that  can  be  evaluated,  but  also  that 
a  system  of  evaluation  which  calls  for  all  deliverables  to  be  evaluated  is  parsimonious 
of  effort  in  the  long  run  and  serves  as  a  systemic  quality  control  program.  The  term 
introduced  here  to  mean  evaluating  all  project  deliverables  as  they  are  completed, 
is  front-to-back  formative  evaluation.  To  illustrate  the  applicability  of  the  proposed 
concept  as  a  prescriptive  theory,  a  graphic  linear  ISD  model  is  presented  which  is 
an  elaboration  of  the  widely  used  Dick  &  Carey  Model.  The  expanded  model,  called 
the  Braden  Formative  Evaluation  Model,  clearly  depicts  the  relationship  between 
formative  evaluation  activities  and  instructional  design  and  development  procedures. 
Predictive  evaluation  is  defined  and  distinguished  from  formative  evaluation,  with 
the  suggestion  that  its  role  in  instructional  design  needs  to  be  fully  elaborated. 
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Formative  Evaluation:  A  Revised  Descriptive  Theory  and  a 

Prescriptive  Model 


The  Trend  Toward  Formative  Process  Evaluation 


In  the  past  decade  or  so,  but  particularly  in  the  past  five  years,  this  author  and 
several  others  have  moved  away  from  a  simple  prescription  for  end-of-project 
formative  product  evaluation  and  toward  more  complex  prescriptions  for  expanded 
formative  evaluation  as  part  ofthelSD  process  (Braden  1987, 1991;  Carey  &  Carey, 
1980;  Flagg,  1990;Nervig,  1990;  Reeves,  1989;  Van  Patten,  1989).  At  the  same  time, 
however,  there  has  been  a  continuation  of  highly  respectable  material  written  which 
has  clung  to  the  narrower  conception  of  the  role  of  formative  evaluation  in 
instructional  design  and  development  (e.g.,  Dick  &  Carey,  1990,  1991;  Russsell  & 
Blake,  1988). 

A  classic  contribution  to  the  unfolding  of  a  broader  definition  of  formative 
evaluation  is  Barbara  Flagg's  Formative  Evaluation  for  Educational  Tedmologies 
(1990).  Although  Flagg  did  not  offer  a  model  for  process-oriented  formative 
evaluation  per  se,  she  did  identify  four  phases  of  formative  evaluation.  Taken 
together  these  phases  represent  a  front-to-back  implementation  of  formative 
evaluation  methods.  Flagg's  phases  were  presented  in  the  following  format: 


Phases  of  Program  Development  Phases  of  Evaluation 

Planning  Needs  Assessment 

Design  Pre-production  Formative  Evaluation 

Production  Production  Formative  Evaluation 

Implementation  Implementation  Formative  Evaluation 

(Flagg,  1990,  p.  35.) 

The  thought  that  all  of  us  evaluate  everything  we  do  as  we  go  along  is  so  logical 
as  to  be  considered  fundamental.  In  that  sense,  suggesting  that  ID  practitioners 
evaluate  the  output  of  each  major  step  or  activity  as  they  proceed  through 
instructional  design  and  development  could  be  interpreted  as  trivial,  gratuitous,  or 
even  insulting.  Therefore,  the  suggestion  of  this  paper  goes  further,  contending  that 
casual  consideration  of  the  quality  of  work-in-progress  is  not  the  same  thing  as 
systematic,  thorough,  on-going  formative  evaluation.  That  said,  the  only  thing  new 
about  front-to-back  formative  evaluation  as  a  part  of  instructional  design  and 
development  is  a  set  of  implementation  ideas:  (1)  Steps  should  be  added  to  ISD 
models  to  assure  that  formative  evaluation  is  both  systematic  and  systemic,  and  (2) 
The  concept  of  formal  front-to-back  formative  evaluation  must  be  moved  from  the 
concept  stage  into  everyday  practice. 
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Models 


A  model  for  the  evaluation  01  Instructional  development  must,  necessarily,  be 
based  upon  'he  instructional  systems  design  (iSD)  process  itself.  Instructional 
systf  im  design  models  are  not  in  short  supply.  Twenty  years  ago  Stamas  (1972) 
wroie  an  occasional  paper  for  the  Division  of  Instructional  Dwelopment  that 
ron.pared  23  ISD  models.  More  recently,  Gustafson  (1991)  considered  33  models 
lid  oi  e  selecting  a  dozen  to  review  in  the  second  edition  of  a  Survey  of  Instructional 
Development  Models.  In  between  times  there  have  been  literally  dozens  of  models 
proposed.  In  a  single  article  Andrews  and  Goodson  (1980)  analyzed  40  instructional 
design  models  selected  from  over  60  they  had  identified.  Even  with  all  of  this 
outpouring  of  models—some  descriptive,  some  prescriptive,  dome  astute,  some 
inane— only  a  few  have  gained  more  than  passing  attention  because  most  of  the 
models  tend  to  be  similar  to  each  other, 

Gropper  (1977)  compiled  a  list  of  10  generally  agreed  upon  steps  found  in  most 
ISD  models.  Andrews  and  Goodson  extended  that  list  to  14  items,  but  conceded  that 
all  four  additions  were  actually  com;  onents  of  a  single  process,  needs  assessment. 
Thus  a  generally  agreed  upon  list  widh  1 1  steps  would  include  tasks  associated  with 
needs  assessment,  specifying  outcomes,  goal/task  analysis,  sequencing  goals/ 
objectives,  identifying  learner  attributes,  formulation  of  strategies,  media  selec'  jn. 
materials  development,  tryout/ revision,  and  install/maintain.  Notice  that  the  term 
evaluation  does  not  appear.  In  actuality,  many  of  the  models  upon  which  the 
consensus  steps  were  based  included  the  term  evaluation  rather  than  tryout.  This 
author  holds  that  the  step  "install/maintain"  is  a  post-design  event  and  therefore 
isn't  really  an  ISD  step— which  would  leave  us  with  10  design  steps  as  building 
blocks  if  we  were  inclined  to  create  an  all  inclusive  generic  ISD  model. 

The  conception  of  a  systems  process  composed  of  a  series  of  steps,  whether  they 
are  those  listed  above  or  others,  lends  itself  to  a  flow  chart  display.  The  principle 
model-design  elements  for  flow  charting  are  the  box  and  the  arrow.  As  a  result,  most 
ISD  models  are  schematically  some  sort  of  elaboration  upon  the  basic  arrow-box- 
arrow  design  of  the  basic  input-process-output  model  (see  Figure  1). 


Process 


Figure  L  Basic  Input-Process-Output  Model 
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A  common  characteristic  of  even  the  earlier  instructional  systems  models  has 
been  the  feedback  loop  (e.g.,  Banathy,  1968).  By  applying  a  feedback  loop  to  the 
basic  input-process-output  model  we  get  something  that  looks  like  Figure  2. 


Input 

Process 

Output 

f  Feedback 

Figure  2.  Basic  Feedback  Model 

Next,  if  we  consider  that  the  source  of  any  useful  data  to  be  fed  back  in  the 
feedback  loop  is  likely  to  be  some  form  of  evaluation  of  the  output,  we  could 
construct  a  more  accurate  depiction  of  the  basic  feedback  model.  For  example  we 
might  designate  a  circle  with  the  letters  F.E.  inside  to  represent  formative  evaluation 
of  the  process  output.  The  graphical  result  would  look  something  like  Figure  3. 


Input 


Process 


[F  E  ] =  Normative 

Evaluation 


I 


Output 


Feedback 


Figure  3.  Basic  Evaluation  Feedback  Model 

Skipping  ahead  to  a  more  complex  graphic  display  of  a  procedure,  we  might  wish 
to  show  several  separate  steps— as.  for  example,  in  a  generic  instructional  systems 
model.  We  must  bear  in  mind,  however,  that  every  box  in  a  flow  chart  model 
represents  its  own  sub-process.  Thus,  if  we  chose  to  do  so  we  could  display  each 
traditional  ISD  macro-step  as  its  own  multiple  step  box-and-arrow  model,  showing 
all  of  the  mini-steps.  Complete  with  feedback  loop,  the  diagrammatic  features  would 
be  similar  to  Figure  4.  Note  that  the  input  arrow  is  implied  rather  than  shown  and 
that  output  has  been  renamed  "implement." 
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Step  2 


Stepn 


Evaluate 


Implement 


Figure  4.  Diagram  of  a  Multi-Step  Generic  ISD  Model 

Figure  5  illustrates  what  happens  to  our  diagram  when  we  consider  that  the  ntire 
instructional  design  procedure  is  nothing  more  than  a  process  which  could  be 
represented  in  a  single  box.  We  have  a  clear  example  of  a  self  correcting  process 
wherein  the  element  of  evaluation  for  feedback  is  the  critical  part  of  the  process 
which  dictates  any  needed  revision. 
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Figure  5.  Input-Process-Output  Model  Combined  with  Generic  ISD  Model 


An  ISD  Model  That  Is  Also  a  Formative  Evaluation  Model 

Front-toBack  Formative  Evaluation 

A  major  purpose  of  this  paper  is  to  propose  an  ISD  model  that  is  also  an  evaluation 
model.  In  simple  terms,  the  key  to  the  model  is  that  each  step  results  in  one  or  more 
deliverables  and  that  each  deliverable  is  formatively  evaluated  and  revised  as 
necessary  before  proceeding  to  the  next  step.  This  concept  of  front- to  back  form*  tive 
evaluation  has  been  used  successfully  by  the  author  in  practice  and  in  his 
classroom  for  several  years  (Braden,  1987).  The  ISD  model  which  was  proposed  by 
Van  Patten  (1989)  also  calls  for  front-to-back  formative  evaluation,  and  isalsobased 
upon  the  premise  thateach  deliverable  should  be  specified,  created,  then  formatively 
evaluated.  Earlier,  Carey  &  Carey  (1980)  suggested  a  general  set  of  formative 
evaluation  questions  the  answers  to  which  would  assist  local  selection  committees 
in  reviewing  instructional  materials.  The  Carey  &  Carey  approach  was  comprehensive , 
dealing  with  all  aspects  of  the  instructional  design  process,  but  they  did  not  suggest 
that  each  sub-process  (step)  be  formatively  evaluated  before  proceeding  to  the  next 
step. 
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A  Symbol  for  Formative  Evaluation  of  Deliverables 

A  definition  of  the  symbolic  notation  used  by  this  author  in  a  front-to-back 
formative  evaluation  model  is  critical  to  a  full  understanding  of  that  model  and  its 
implications.  Specifically,  it  is  essential  that  readers  know  the  meaning  of  a  symbol 
made  up  of  a  rectangle  with  an  embedded  circle  containing  the  letters  F.E.  In  short, 
it  means  Normative  evaluation  is  the  last  sub-step  performed  in  that  step  of  the 
model.H  Thus  a  box  with  an  embedded  F.E  circle,  although  symbolically  different, 
is  identical  in  meaning  to  the  box — arrow— -F.E.Circle — feedback-loop  diagram 
shown  in  Figure  3.  In  essence  the  new  symbol  represents  any  process  or  procedure 
that  has  a  built-in  evaluation  and  revision  provision,  making  It  self-correcting  (see 
Figure  6). 


Input 


Figure  6.  A  New  Icon:  The  Symbol  for  a  Self-Correcting  Procedure 

The  Elaborated  Instructional  Design  Model 

Revising  the  Structure  of  the  Consensus  ISD  Model 

Although  the  proliferation  of  instructional  design  models  has  slowed  down  from 
the  frantic  pace  of  the  1970's  (Gustafson,  1991),  between  the  "not  invented  hereM 
syndrome  and  the  insatiable  need  of  certain  personality  types  to  tweak  and  polish, 
we  can  expect  several  new  models  each  year.  As  Shrock  has  observed,  during  the 
1970*8,  "experience  with  instructional  development  was  revealing  problems,  and 
important  and  permanent  modifications  of  the  earlier  models  were  made.  One  of  the 
most  important  was  the  addition  of  needs  assessment  to  the  collection  of  steps  that 
defined  the  process."  (1991,  p.  17.) 

Three  other  important  modifications  could  become  generally  accepted  during  the 
1990's:  elaboration  of  the  step  labeled  "strategies"  (Leshin,  Pollock  &  Reigeluth, 
1992),  accommodation  of  the  concurrent  design  of  instruction  and  motivation 
(Keller,  1979,  1983),  and  integration  of  front-to-back  formative  evaluation  (Braden, 
1987;  Van  Patten,  1989).  Adding  motivation  to  the  elaborated  model  and  expanding 
the  strategies  component — while  called  for— are  not  the  purposes  of  this  paper.  So 
those  addenda  will  be  left  to  another  time. 

Adding  Front-to-Back  Formative  Evaluation  Points  to  the  ISD  Model 

Elaborating  the  model  with  front-to-back  formative  evaluation  is  on  the  current 
agenda,  and  Figures  7a  and  7b  gr  aphically  illustrate  how  easily  the  modification  can 
be  n  ade.  To  facilitate  discussion,  the  elaborated  model  (Figure  7b.)  has  been  labeled 
the  Braden  Model  to  distinguish  it  from  its  obvious  predecessor,  the  DJck  &  Carey 
Model  (Figure.  7.  a.).  The  Dick  &  Carey  Model  is  one  of  the  three  or  four  best  known 
ISD  models,  and  the  text  in  which  it  is  featured  is  now  in  a  third  edition  (Dick  & 
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Carey,  1990).  That  particular  model  was  selected  for  elaboration  because  of  its 
recognition  value.  The  original  version  of  the  elaborated  Dick  &  Carey  Model  was 
first  shown  at  the  1987  DID  luncheon. 


In  its  present  form  the  Braden  Model  departs  from  the  Dick  &  Carey  Model  in  three 
significant  ways.  First,  and  most  obvious,  the  T.E."  nodes  have  been  added. 
Second,  an  "organize  management"  box  has  been  added  which  includes  writing  the 
evaluation  plan.  Since  management  functions  are  apart  from  the  design  process, 
that  box  is  displayed  "off  line."  Third,  the  step  which  Dick  and  Carey  label  "Design 
and  Conduct  Formative  Evaluation**  has  been  more  appropriately  identified  as 
"Conduct  Prototype  Test."  That  label  was  chosen  in  preference  to  "Tryout"  because 
it  was  felt  that  "prototype"  emphasized  the  formative  nature  of  the  instructional 
materials  until  after  they  "pass*  the  test. 

•  Formative  Evaluation  (RE.)  Nodes 

A  characteristic  of  each  box  in  the  Dick  &  Carey  Model  is  that  the 
box  represents  a  discrete  sub-process.  An  argument  could  be  made 
that  some  or  all  of  the  sub-processes  could  be  divided  and  displayed 
as  two  or  more  boxes  each,  but  that  is  irrelevant.  What  matters  is  that 
the  end  of  eveiy  sub-process  shown  is  distinct,  and  is  observable  in 
the  form  of  a  tangible  deliverable(s).  Two  factors  were  considered  in 
adding  these  nodes:  (1)  Anything  that  is  tangible  can  be  evaluated. 
(2)  In  a  linear  process  where  linearity  is  a  required  condition,  each 
step  is  a  prerequisite  for  the  one(s)  that  follow,  in  effect,  the 
deliverable  or  output  of  step  one  is  all  or  part  of  the  input  of  step  two. 

The  most  radical  aspect  of  the  front-to -back  formative  evaluation 
procedure  implied  in  the  F.E.  nodes  is  that  there  is  a  philosophical 
departure  from  the  developmental  testing  of  instruction  (not  the 
procedure  for  designing  that  instruction,)  Walter  Dick,  the  most 
frequently  cited  author  on  the  subject  of  formative  evaluation  of 
instructional  design,  set  a  narrow  focus  for  the  field  when  he  wrote, 
"These  developmental  testing  procedures  now  tend  to  be  considered 
as  the  operational  definition  of  formative  evaluation..."  (Dick,  1980, 
p.3.)  In  terms  of  the  types  of  evaluation  defined  by  Stuffelbeam 
(197o),  that  operational  definition  limits  formative  evaluation  to 
product  evaluation.  Inherent  in  the  step-at-a-time  evaluation  scheme 
of  the  Braden  Model  is  the  opportunity  to  utilize  evaluation  data  for 
the  puipose  of  making  appropriate  alterations  to  the  process.  [In 
fairness  to  Dick,  he  was  absolutely  correct  A  limited  concept  of 
formative  evaluation  had  beengenerally  accepted.  The  tone  was  likely 
set  by  Bloom,  Hastings  andMadaus  who  authored  a  large  format  book 
of  over  900  pages  onformative  and  summative  evaluation.  Less  than 
two  pages  ofttiat  tome  were  devoted  to  the  prospect  of  using  formative 
evaluation  as  the  basis  of  altering  the  process  and  for  the  companion 
purpose  of  process  quality  conti  ul  (see  Bloom.  Hastings.  andMadaus. 
1971.  pp.  135-6.)] 
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•  Organic  Management 

Project  management  has  been  a  concern  of  instructional  designers 
for  two  decades. One  of  the  nine  steps  in  the  widely  disseminated 
Instructional  Development  Institute  (IDI)  Model  is  "Organize 
Management*  (N. M.S. I.,  1971).  In  addition  to  logistics  planning,  time 
allocation,  budgeting,  and  so  forth,  a  critical  management 
consideration  is  quality  control.  The  heart  of  quality  control  is 
evaluation,  and  flaws  caught  early  avoid  compounding  of  errors 
which  is  desirable  because  simple  errors  are  more  easily  remedied. 
Greer  has  recently  brought  new  light  to  this  often  ignored  aspect  of 
instructional  design.  In  a  chapter  about  ID  project  management 
(Greer,  1991),  he  listed  eight  "inspection  points"  in  the  typical  ID 
project.  He  followed  that  work  with  a  textbook  (Greer,  1992)  on  the 
same  subject  which  included  his  version  of  an  ISD  model.  In  the 
Greer  Model,  phase  I  is  "Project  Planning,"  which  coincides  closely 
with  the  front-of-the-model  location  given  to  management  planning 
in  the  Braden  Formative  Evaluation  Model. 

•  Conduct  Prototype  Test 

Except  for  the  change  in  terminology,  there  is  no  difference  in  what 
happens  at  this  step  in  the  Dick  &  Carey  Model  and  in  the  Braden 
Model.  Dick  and  Carey  (1990,  1991)  have  done  a  splendid  job  of 
explaining  the  purposes  and  techniques  of  formative  product 
evaluation  as  it  applies  to  the  instructional  materials  resulting  from 
an  instructional  design  project.  Unquestionably  their  description  of 
what  is  to  occur  at  this  step  is  a  formative  evaluation  activity. 
However,  the  activity  at  that  Juncture  is  only  a  part  of  the  total 
formative  evaluation  of  the  project,  and  therefore  deserves  a  more 
accurate  label:  tryout  or  prototype  test. 


Imposing  Front-to-Back  Formative  Evaluation  upon  Existing  ISD 

Models 

Not  every  ISD  model  can  readily  be  adapted  to  incorporate  front-to-back  formative 
evaluation.  However  the  majority  or  ISD  models  can  be  so  adapted  with  ease.  The 
requirements  for  applying  the  f-t-b  formative  evaluation  component  to  an  existing 
ISD  model  are: 

1 .  The  model  should  be  linear.  [Non-linear  models  might  benefit  also,  but  a  major 

rationale  for  adding  f-t-b  F.E.  would  be  lost.] 

2.  The  model  should  be  a  prescriptive  model  for,  not  a  descriptive  model  of. 

(e.g.,  a  grapitk:  model  for  is  the  Dick  &  Carey  Model  (1990)  which 
prescribes  a  sequence  of  design  activities.) 


ERLC 


079 

yy 


[e.g.,  a  textual,  non-graphic  model  of  is  Tennyson's  Instructional 
Design  Model  (1990)  which  describes  the  linkage  between  cognitive 
learning  theory  and  instructional  prescriptions,] 

[e.g.,  a  graphic  model  of  is  Tennyson's  Model  of  Cognitive  System  for 
Educational  Learning  Theory  (1992),  which  uses  boxes  and  arrows, 
but  uses  them  more  to  describe  interactions  than  to  prescribe  linear 
function,] 

How  it  Works 

A  fair  question  at  this  point  would  be,  w0Kf  just  what  do  you  do  in  a  front-to-back 
formative  evaluation?'*  A  short  answer  would  be  that  at  every  F.E,  node  in  the 
process  you  ask  questions  of  consistency  and  adequacy,  (For  the  origin  of  that 
method  of  assessing  step-to-step  quality,  see  Reigeluth's  1980  article  on  The 
Instructional  Quality  Profile,)  In  reality  the  full  answer  *s  mi  ~,h  more  complex.  As 
with  any  evaluation  effort,  the  exact  methods  depend  upon  the  nature  of  the  data 
we  wish  to  collect  At  the  highest  level  of  generalization,  let  us  theorize  how  we  might 
perform  what  is  called  for  at  an  F.E.  node.  First,  we  would  start  with  a  deliverable 
(a  product).  Next  we  would  ask  questions  of  consistency  such  as:  Is  it  appropriate? 
Does  it  reflect  the  input  from  the  previous  step?  Is  it  consistent  with  the  project 
goals,  the  performance  objectives,  etc.?  We  would  then  gather  whatever  data  we 
deemed  necessary  to  answer  those  questions  to  our  satisfaction.  Then ,  we  would  ask 
questions  of  adequacy  such  as:  Is  this  sufficient?  Will  this  deliverable  be  useable  as 
the  input  for  the  next  step  in  the  ID  process?  Does  this  product  reflect  all  of  the 
elements  of  the  input,  te.,  is  it  complete?  Again  we  would  choose  the  data  gathering 
technique  which  best  matched  the  circumstances  and  the  nature  of  the  information 
sought.  With  all  of  the  data  in  hand  we  would  assess  the  quality  and  ongoing  utility 
of  the  deliverable.  In  the  event  that  the  evidence  indicated  that  the  deliverable  was 
either  inappropriate  or  insufficient,  we  would  pose  a  different  kind  of  question:  What 
caused  the  flaw  or  inadequacy?  The  search  for  that  answer  invites  process 
evaluation.  We  want  to  know  what  we  did  wrong,  what  we  did  poorly.  If  no  process 
errors  are  detected,  we  would  then  loop  back  to  the  previous  step  to  re-examine  its 
output. 


The  Time  Factor 

Another  fair  question  would  be,  "Isn't  adding  all  of  that  routine  checking  of  the 
deliverables  too  time  consuming?"  Again  a  short  answer  would  be  that  if  you  never 
make  mistakes,  it  is  too  time  consuming.  Otherwise,  catching  flaws  and  errors  early 
tends  to  save  time.  Once  you  have  prepared  a  set  of  checklists,  routines,  and 
questions  to  be  used  at  each  node,  you  will  find  that  there  is  a  high  level  of  transfer 
of  those  tools  to  your  next  project.  Armed  with  a  set  of  evaluation  instruments  that 
need  little  or  no  revision,  the  task  requires  much  less  of  your  time. 

Looking  at  the  other  addition  that  was  made  to  the  original  Dick  &  Carey  Model, 
a  parallel  question  might  be  asked  about  the  time  consumed  in  the  writing  of  an 
evaluation  plan.  The  short  answer  to  that  is  to  be  found  in  the  management  value 


ERLC 


080 

lUu 


of  the  evaluation  plan  itself.  Just  knowing  that  everything  will  be  put  *o  the  test 
establishes  a  mind  set  for  high  standards  and  excellence.  Knowing  that  the  building 
blocks  of  the  present  ID  event  have  been  sharply  scrutinized  adds  confidence  that 
the  project  is  on  track.  Detection  of  problems  that  require  remediation  provides  early 
warning  that  the  time  line  for  the  project  is  in  Jeopardy,  allowing  the  designer  more 
time  to  renegotiate  delivery  dates  or  to  bring  more  resources  to  bear  on  the  project. 
The  data  gathering  also  has  implications  for  matters  of  personnel  management, 
budget  monitoring,  avoidance  of  political  problems,  and  all  kinds  of  other  factors 
where  information  might  be  important  to  decision-making.  Is  the  time  spent  worth 
all  of  these  benefits?  In  most  cases,  yes.  Besides,  one  evaluation  plan  tends  to 
resemble  all  others.  Just  as  there  is  a  carryover  of  tools  for  the  F.E.  of  deliverables, 
there  >s  a  carryover  from  evaluation  plan  to  evaluation  plan.  After  you  have  once 
assembled  a  plan  that  you  like,  large  portions  of  it  can  be  used  as  "boiler  plate"  in 
the  writing  of  your  next  evaluation  plan. 


Being  Consistent  with  Stuffelbeam's  CIPP  Model 

The  literature  of  evaluation  is  permeated  with  references  to  Stuffelbeam's  CIPP 
Model  (Stuffelbeam,  1973).  Perhaps  the  most  significant  characteristic  of  CIPP  is 
that  it  makes  provision  for  holistic  evaluation.  Its  elements  are  systems  oriented, 
structured  to  accommodate  universal  evaluation  needs.  The  tenets  of  CIPP  are  so 
widely  accepted,  so  firmly  rooted  that  they  cannot  be  ignored.  With  that  in  mind,  a 
reasonable  course  of  action  would  be  to  use  CIPP  as  a  litmus  test  for  any  serious 
proposal  of  an  evaluation  system.  Then,  assuming  that  we  consider  the  amount  and 
nature  of  the  evaluation  encompassed  within  the  instructional  design  process  to 
constitute  uan  evaluation  system "  we  can  [should?]  ask,  "How  does  this  system 
stack  up  in  terms  of  CIPP? 

•  Context  is  evaluated  in  needs  assessment  when  we  evaluate  the  status 

quo.  Flagg(1990)  adopted  the  term  needs  assessment  as  the  label  for 
the  first  of  her  four  phases  of  formative  evaluation.  In  the  model 
offered  here,  the  evaluation  of  goals  implies  a  consideration  of 
whether  it  is  appropriate  to  plan  to  employ  instruction  as  the 
appropriate  means  to  improve  the  status  quo  (the  context). 

•  Input  is  evaluated  in  both  needs  assessment  and  during  student  analysis 

when  we  evaluate  the  students'  entry  competencies.  Additionally, 
there  is  an  element  of  input  evaluation  in  what  we  do  while  assessing 
subject  content  during  instructional  goal/ task  analysis. 

•  Process  evaluation  is  not  prescribed  in  the  vast  majority  of  ISD  models. 

Front-to-back  F.E.  as  proposed  herein  does  prescribe  process 
evaluation.  (Formative  evaluation  of  the  deliverable  of  any  given  ISD 
process  step  is  product  evaluation  initially.  However,  as  soon  as  a 
flaw  in  the  product  is  determined,  the  procedure  shifts  to  an 
evaluation  of  the  process  which  permitted  the  flaw  to  occur.)  None 
of  the  ISD  models  reviewed  in  Gustafson  (1991),  Stomas  (1970), 
Gropper  (1977),  or  elsewhere  include  a  stepfc)  for  process 
evaluation.  Except  for  the  model  proposed  here,  only  the  Van  Patten 
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Model,  and  the  Flagg  phases  (both  non-graphic)  provide  for  process 
evaluation.  Yet  Reeves  (1989,  p.  164.)  recognized  that  process 
evaluation  was  needed:  "Obviously,  prototype  products  will  be  a 
major  focus  of  formative  evaluation  during  an  instructional  design 
project,  but  there  should  be  an  equal  emphasis  on  collecting 
formative  evaluation  to  improve  the  overall  mstructional  design 
process..."  (Emphasis  mine.) 

•  Product  evaluation  describes  what  happens  in  the  ISD  tryout  step. 
Although  referred  to  in  the  Dick  &  Carey  and  several  other  models  as 
formative  evaluation,  and  in  many  models  simply  as  evaluation, 
the  term  formative  product  evaluation  more  accurately  describes 
the  tryout  process.  The  term  prototype  test,  as  used  in  the  model 
offered  in  this  paper,  is  also  a  product  evaluation  label  that  accurately 
describes  the  procedure.  As  mentioned  in  the  discussion  of  process 
evaluation,  the  F.E.  nodes  in  the  Braden  Model  are  product  evaluation 
points  until  a  deficiency  or  product  flaw  is  detected. 


Predictive  Evaluation 

The  terms  formative  and  summative  evaluation  have  been  around  long  enough 
now  that  they  are  part  of  the  vocabulary  of  every  instructional  designer  and 
developer.  In  a  black  or  white  world  those  two  forms  of  evaluation  would  represent 
the  entire  palette  of  evaluation  choices.  Things  are  not  as  simple  as  black  and  white, 
however,  and  even  though  our  palette  isn't  a  full  spectrum  rainbow,  it  does  contain 
some  tints  and  shades.  Most  notable,  of  course,  is  Needs  Assessment  which  fits  into 
neither  the  formative  crate  nor  the  summative  barrel. 

One  way  to  view  the  kinds  of  evaluation  that  we  need  in  ISD  is  to  consider  the 
underlying  purposes  of  each. 

•  We  use  discrepancy  analysis  evaluation  (including  needs 

assessment)  for  establishing  parameters  during  goal 
settlngand  some  forms  of  decision  making.  Any  decision 
that  requires  a  choice  will  benefit  from  this  kind  of 
evaluation  of  the  things  that  might  be  chosen. 

•  We  use  formative  evaluation  for  quality  control.  Either  the 

process  or  the  associated  products  can  be  the  focus  of 
formative  evaluation. 

•  We  use  summative  evaluation  for  accountability  purposes. 

Sometimes  we  perform  summative  evaluation  just  to 
satisfy  our  curiosity  about  how  effective  a  product  has 
been,  but  more  likely  needs  for  summative  evaluation 
are  linked  to  costs  vs.  benefits  analysis  and  to  decisions 
about  program  funding  or  survival.  Summative 
evaluation  is  something  that  happens  to  instructional 
design,  not  something  that  is  a  part  of  it. 
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•  And  then  there  is  that  other  kind  of  evaluation  —  the  kind 
we  use  to  improve  our  chances  of  being  correct  when  we 
make  a  guess.  For  lack  of  a  better  term,  let's  call  this 
kind  of  evaluation  "predictive  evaluation." 

Instructional  designers  do  a  lot  of  predictive  evaluating  when  they  organize  the 
management  of  projects.  Budget  estimation,  time  estimation,  guesses  about 
materials  that  will  be  used  and  the  skills  that  will  be  required  are  all  imprecise 
activities.  Intuitively  designers  attempt  to  improve  their  guessing  average  by  basing 
their  estimates  upon  good,  analyzed  information.  The  art  of  forecasting  will  always 
contain  a  luck  factor,  but  luck  <-an  be  reduced  by  evaluating  the  options  open  to  us. 
A  method  to  assist  us  with  the  evaluative  aspects  of  estimation  would  greatly  benefit 
the  instructional  design  and  development  field. 

We  are  likely  soon  to  find  that  predictive  evaluation  has  become  as  much  a  part 
of  ISD  as  formative  evaluation  —but  in  the  form  of  an  expert  system  rather  than  as 
a  box  in  a  model.  If  not,  w*  will  need  to  accommodate  it  in  our  evaluation  models, 
if  not  in  the  ISD  models  t  jmselves. 
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Abstract 

The  hypothesis  that  the  planning  activities  of  classroom  teachers  correlate  with 
the  practices  of  instructional  design  professionals  is  explored  within  the  context 
of  this  study.  Classroom  teachers  participated  in  a  survey  which  requested 
information  regarding  their  planning  routines,  A  35-item  two  part  questionnaire 
was  used  as  the  data  collection  instrument,  Sixty-one  public  school  teachers 
reported  on  their  actions  when  planning  to  teach  on  a  daily  basis,  and  indicated 
such  demographic  information  as  typical  c  lass  size,  number  of  years  teaching, 
grade  level  and  educational  background,  Results  indicate  that  a  correlation  exits 
between  teacher  planning  activities  and  instructional  design  practices,  and  that 
such  a  relationship  is  influenced  by  the  subject  taught. 

Preliminary  investigations  of  teachers  as  instructional  designers  (Branch,  1986;  Branch,  Darwa/eh  & 
El-Hindi,  1991)  suggest  there  is  a  false  assumption  that  teachers  do  not  engage  in  instructional  design 
practices,  Further,  instructional  design  jargon  tends  to  inhibit  communication  between  instructional  design 
professionals  and  public  school  teachers.  In  addition,  it  appears  that  certain  teacher  profiles  may  enhance 
the  potential  for  teachers  to  practice  instructional  design  or  that  the  contextual  factors  which  encompass 
public  school  environments  make  the  use  of  traditional  instructional  design  models  impractical. 

This  called  for  a  shift  of  the  focus  on  teachers  as  instructional  designers  away  from  the  perspective  of 
the  instructional  designer  and  more  toward  the  perspective  of  the  public  school  classroom  teacher.  In  order 
to  accomplish  this  there  was  a  need  to  identify  teacher  planning  practices  and  instructional  design  practices 
that  arc  essentially  similar  in  purpose  and  orientation,  but  different  in  execution.  This  also  prompted  a  need 
for  identifying  correlations  between  the  potential  for  teachers  to  practice  instructional  design  and  contextual 
factors  such  as  number  of  years  teaching,  class  size  and  education  level  of  the  teacher. 

This  is  a  report  of  an  investigation  which  sought  to  answer  the  fundamental  question:  Do  teachers 
practice  instructional  design?  from  the  perspective  of  public  school  teachers.  This  study  was  conducted 
based  on  teacher  planning  research  conducted  during  the  past  decade  which  suggests  that  teachers  actively 
develop  routines  which  are  executed  in  the  school  environment  (Apple field  &  Eorle,  1990;  Earlc,  198?; 
Kerr,  1981;  Sherman,  1978;  Zahorik.  1975),  The  process  for  developing  teacher  planning  routines  has 
evolved  from  experienced  educational  practices  as  well  as  from  current  teaching  theories  and  learning 
theories.  Instructional  designers  systematically  select,  adapt,  develop  and  refine  a  wide  varietv  of 
instructional  products  (Martin.  1984),  The  process  of  instructional  design  evolved  from  a  conceptual 
amalgam  of  general  systems  theory  ( Banathy,  1968;  1973),  the  application  of  tcchnoloav  to  educational 
methodology  (Chisholm  &  Ely,  1976).  and  the  psychology  of  learning  (Gaelic.  1977) "  A  review  of 
instructional  design  procedures  and  teacher  planning  routines  literature  (  Andrews  &  Goodson,  I  WO:  Brigus 
&  Wager,  1981;  Dick  &  Carey  1990;  Gagne,  Briggs.  &  Wager  1992;  Merrill  1983:  Merrill,  Reigeluth/& 
Faust  1979;  Pratt,  1980:  Reigeluth.  &  Stein,  1983:  Earlc.  1985;  Yingcr.  1979;  Zahorik.  1975)  reveals  that 
teachers  and  instructional  designers  are  involved  in  similar  basic  activities  such  as:  planning,  designing 
lesson  plans,  designing  unit  plans,  designing  test  items,  stating  objectives,  managing,  evaluating  "and 
consulting.  Instructional  designers  tend  to  focus  on  selecting  instructional  materials,  analyzing  content, 
sequencing  content,  and  decision-making;  whereas,  teachers  focus  primarily  on  the  implementation  and 
evaluation  of  instruction. 

Because  teachers  are  at  the  "front  line"  in  the  educational  process  (Earle,  1985),  understanding  the  logic 
of  their  actions  is  important  as  educators  attempt  to  increase  the  efficiency  of  the  educational  process. 
Beilby  (1974)  emphasized  the  importance  of  instructional  development  for  teachers  by  questioning  the  view 
that  the  instructional  design  process  must  be  managed  by  the  education  specialist,  Beilby  (1974)  presented 
a  strong  case  for  teacher  involvement  in  instructional  design  and  stressed  the  need  for  training  teachers  for 
the  instructional  designer  role.  In  the  past  teachers  have  developed  instructional  design  skills  through  I )  the 
pre-scrvice  curriculum,  2)  in-service  training,  and  3)  working  with  an  instructional  designer  to  improve 
present  courses.  While  the  second  two  needs  are  important,  it  is  the  first  need,  the  pre-service  teacher 
preparation  curriculum,  this  project  addresses  by  developing  an  inventory  of  teacher  planning  information 
that  focuses  on  instructional  designer  competencies  which  might  be  incorporated  into  teacher  preparation 
programs. 

The  rationale  for  this  investigation  is  that  successful  teachers  engage  in  similar  actions  to  those 
associated  with  instructional  design  when  preparing  to  teach.  But.  does  this  mean  teachers  are  instructional 
designers?  According  to  Kerr  ( 198 1 );  Teachers  arc  and  are  not  instructional  designers.  Most  teachers  have 
not  had  formal  training  in  the  procedures  commonly  used  by  instructional  designers:  many  find  it  difficult 
to  shift  their  thinking  into  instructional  design  (ID)  patterns  when  they  are  asked  to  do  so  as  part  of  a  course 
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or  workshop"  (p.  364).  r  <  .     be  partly  attributed  to  ihe  less  than  positive  attitude  of  some  teachers 
toward  the  use  of  instir   .ukh  Vsign  models  out  of  fear  that  a  systematic  approach  restricts  creativity 
(branch,  1986).  Instruct.,  i  osign  ideology  adopts  the  position  thai  a  systems  approach  promotes 
creativity  by  increasing  the  number  of  alternative  generated  and  by  testing  instructional  options  prior  to 
implementation* 

Traditionally,  educators  and  instructors  have  determined  the  major  elements  of  the  instructional  process 
to  be  students*  teachers,  and  curriculum.  Since  1970,  however,  rapid  technological  advances  have 
necessitated  bringing  the  instructional  designer  into  the  core  of  the  instructional  process.  The  systematic 
design  of  instruction  is  usually  considered  to  be  the  role  of  the  instructional  designer.  There  is  a  trend 
among  instructional  design  professionals  however,  to  focus  attention  on  the  need  for  preparing  and  training 
public  school  teachers  for  the  instructional  designer's  role  (Applefield  &  Earle,  1990;  Dick  &  Reiser, 
1989;  Earle,  1985). 

The  contention  is  that  there  is  a  correlation  between  teacher  planning  routines  and  what  instructional 
designers  do  when  designing  instruction.  In  order  to  support  this  bel'ftf,  a  systematic  investigation  is  being 
conducted  to  determine  the  extent  of  this  correlation. 
Instructional  Episode 

Instruction,  as  subsumed  under  the  concept  of  curricul  jm  (Reigcluth,  1983).  is  the  intervention  that 
occurs  during  a  contenl-mcdia-teacher  interaction  where  the  expressed  goal  is  to  facilitate  the  progression  of 
a  learner  from  poirt  A  tu  point  B  along  the  educational  continuum.  An  accurate  assessment  of  learner 
characteristics,  a  thorough  analysis  of  content  attributes,  and  the  potential  of  the  teacher  to  facilitate  the 
learning  prou    are  nc  independent  variables  within  the  instructional  environment  that  the  designer  or 
teacher  attempts  to  manipulate  in  order  to  create  or  improve  instruction.  This  specific  period  of  interaction 
is  hereafter  referred  to  as  the  instructional  episode. 

The  instructional  episode  serves  as  the  formal  educational  vehicle  that  enables  the  learner  to  construct, 
or  reconstruct,  personal  understandings,  values,  and  beliefs.  Instruction  is  complex  because  it  occurs  within 
a  paradigm  where  each  participating  entity  is  within  itself  complex.  Interaction  between  the  Learner,  the 
Content,  the  Media,  the  Teacher  Function,  and  the  Context  within  which  learning  is  to  occur  during  a  given 
period  of  Time  form  the  instructional  episode  (CMT  paradigm).  Considering  all  the  interrelationships  of 
the  instructional  episode  causes  the  complexity  of  the  learning  process  to  increase  exponentially.  Yet.  it  is 
within  this  paradigm  that  all  instruction  occurs  (Figure  1). 
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Within  the  CMT  paradigm,  the  learner  is  the  focus  of  instruction.  Learners  as  individuals,  bring 
multiple  expectations,  goals  and  diverse  values  to  the  classroom  that  affect  how  they  interact  during  an 
instructional  episode,  and  ultimately,  how  tney  learn.  It  is  al*.  •  important  for  instruction  to  account  for  ihe 
individual  differences  'vithin  the  wider  learner  audience. 

Content  as  a  body  of  knowledge  may  be  sialic  or  dynamic,  but  it  is  not  a  collection  of  unorganized 
facts  existing  in  relative  isolation  one  from  Ihe  other  (West,  Fensham  &  Gerrad,  1985).  Rather,  there  are 
certain  orderly  slfuciuie.i  inherent  within  subject  matter  knowledge  (Reigeluth,  Merrill  &  Buiklerson,  1978; 
Wilson,  1985).  Information  is  not  discrete,  isolated  bits  of  knowledge,  bul  exists  within  an  organized 
structure  (hylon  &  Reif,  1984).  Such  structures  reflect  the  interconnections  or  interrelationships  among  the 
facts,  concepts,  and  principles  that  make  up  subject  matter  knowledge  (Anderson,  1983),  Representations 
of  information  and  interrelationships  among  information  can  be  called  knowledge  structures  (Hannum, 
>Q88). 

Media  is  the  vehicle  by  which  teachers  stimulate,  motivate,  illustrate,  provk1^  concrete  experiences, 
and  direct  attention.  Mediated  instruction  which  is  related  to  the  personal  experiences  of  the  learner  will 
assist  in  building  upon  her  or  his  prior  knowledge  and  promote  achievement,  whereas  messages 
communicated  by  various  forms  of  instructional  media  that  is  foreign  to  the  experiences  of  a  student  or 
offensive  to  the  values  of  a  student  tends  to  inhibit  learning. 

Generally,  the  leather  functions  as  decision  maker  regarding  what  will  be  taugnt,  when  it  will  be 
laugh;,  and  how  it  will  be  taught.  The  teacher  is  often  the  primary  information  source  unless  the  teacher 
function;;  solely  a-  i  facilitator  of  information.  As  facilitator,  the  teacher  arranges  classroom  interactions  so 
as  lo  motivate  learners  and  to  guide  the  learner  to  different  ways  of  knowing,  Teachers  direct  learner 
actions,  monitor  progress,  and  manage  the  implementation  of  instruction. 

The  context  within  which  instruction  occurs  directly  and  indirectly  influences  all  decisions  regarding 
any  instructional  episode.  Instruction  does  not  occur  in  a  vacuum  and  failing  to  acknowledge  such  has  a 
potential  for  a  dangerous  misunderstanding  of  the  complexity  of  learning  environments.  The  instructional 
context  inuudes  all  the  conditions  which  designers  and  teachers  should  consider  in  some  regard  to  enhance 
learning.  Certain  contextual  arrangements  can  be  manipulated  by  designers  and  teachers  while  other 
contextual  arrangements  are  beyond  the  Kalm  of  manipulation  by  either  the  designer  or  the  teacher. 

Learner  achievement  of  prespecified  outcomes  depends  on  Ihe  chosen  instructional  stralcgv  for  a 
predetermined  amount  of  time.  such  as  during  a  class  period  or  field  experience.  An  instructional  strategy 
will  vary  in  effectiveness  depending  on  the  allotted  lime.  Moreover,  thi  quality  of  an  instructional  activity 
s  affected  by  the  amount  of  time  that  can  be  devoted  lo  it.  Fuller,  the  abilities  of  individual  learners 
directly  influence  the  amount  of  time  required  to  achieve  specific  learnug  outcomes.  Tunc,  context,  teacher 
function,  media,  content  and  the  learner,  interact  simultaneously  to  form  ihe  complexities  of  instruction. 
Instructional  Design 

As  a  discipline,  instructional  design  is  concerned  with  understanding  and  improving  one  aspect  of 
education:  the  process  of  instruction  (Reigeluih,  1983).  Instructional  designers  have  as  their  principle 
objective  lo  induce  targeted  learners  to  perform  in  prespecified  ways,  They  achieve  results  by  developing 
and  implementing  documented  and  replicable  procedures  for  organizing  the  conditions  for  learning:  and  by 
defining  and  measuring  the  accomplishments  of  instructional  design  in  terms  of  learner  performance 
(Burkman,  1987).  The  optimum  effects  of  how  student  learning  is  facilitated  and  what  actually  occurs  in 
ihe  classroom  environment  determines  what  is  done  during  the  design  of  instruction  which  is  different  from 
what  is  done  during  the  development  of  instruction.  In  addition,  the  role  of  evaluation  in  instruction  and 
the  management  of  activities  associated  with  all  aspects  of  instruction  is  concepiu  Hy  and  practically 
different.  For  the  context  of  this  study,  descriptions  of  design,  development,  evaluation,  and  management 
as  outlined  in  the  "Domains  of  Instructional  Techtu  logy"  (Association  lor  Educational  Communications 
and  Technology,  in  press)  are  used  as  a  reference.  Interpretations  of  those  domains  are  presented  below. 

Instructional  Desig'.i  is  the  planning  phase  of  the  instructional  creation  process.  Instructional 
design  is  descriptive,  such  as  the  presentation  ol  natural  or  existing  interrelationships  that  constitute  a 
content  area.  Instructional  design  is  also  prescriptive,  It  recommends  organization,  or  reorganization,  of 
information  or  a  sequence  of  events  based  on  known  learner  characteristics,  content  as  a  knowledge 
structure,  specific  media  and  their  attributes,  salient  features  of  the  teacher  function,  educational  context,  and 
time  available.  Design  aspects  apply  systems  theory  to  address  the  complexity  of  the  variables  within  the 
CMT  paradigm,  and  organizes  their  interactions  in  intentional  ways, 

Instructional  development  N  ihe  process  of  producing  from  a  detailed  plan  (design)  the  procedures 
and  media  which  support  an  instruction.*'!  episode.  During  the  development  phase,  the  instructional 
procedures  and  media  are  created  and  tested  based  on  the  performance  objectives  identified  in  the  design. 
Instructional  strategies  prescribed  in  the  design  plan,  as  well  as  selection  strategies  and  presentation 
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methodologies,  are  confirmed  or  revised  based  on  the  results  of  several  appropriate  ittraiicns  of  testing 
throughout  the  various  stages  of  development.  Innovative  and  traditional  technologies  are  employed  to 
conduct  development  activities. 

Instructional  Evaluation  is  a  dynamic  process  10  obtain  data  about  how  students  learn  specific 
content  information  under  varying  instructional  conditions.  These  data  are  analyzed  and  synthesized  into 
ways  raid  means  that  are  used  forjudging  the  instruct  lonaJ  potential  of  the  planned  instructional  episode. 
Evaluation  data  collected  during  the  design  and  development  phases  form  the  tosis  for  revision  of 
instructional  strategies  and  influence  choices  of  instructional  media  prior  to  the  implementation  of  an 
instructional  episode.  Instructional  evaluation  initiates,  permeates  and  concludes  the  instructional  design 
and  development  process. 

Instructional  Management  is  concerned  with  the  supervision  of  instructional  episodes,  including 
pre-episodic  and  post-episodic  activities,  as  well  as  the  human  and  financial  resources  to  support  an 
instructional  episode.  Legislation,  governance,  monitoring  and  certification  of  the  instructional  design, 
development,  and  evaluation  processes  needs  the  endorsement  of  management. 
Teachers  as  Desigrers 

Dick  and  Carey  ( 1990),  and  Gagne,  Briggs,  and  Wager  (1992)  perceived  the  teachers'  role  as  that  of 
designer  of  instruction  with  accompanying  roles  of  implementor  and  evaluator  of  instruction.  Others  have 
taken  the  stance  that  generic  instructional  design  skills  have  value  for  the  classroom  teacher  (Appleficld  & 
Earle,  1990;  Bielby,  1974;  Dick  &  Carey,  1990;  Dick  &  Reiser,  1989),  The  University  of  North  Caiolina 
at  Wilmington,  has  integrated  a  two  semester  instructional  design  component  into  their  undergraduate 
teacher  education  program  (Appleficld  &  Earle,  1990).  Clearly  the  roles  of  classroom  teachers  are  like  that 
of  instructional  designers,  In  fact,  taking  on  the  role  of  instructional  designer,  on  the  part  of  public  school 
teachers  would  have  a  great  influence  on  the  quality  of  the  teachers'  professional  performance,  and  hence, 
on  the  level  of  their  students'  academic  achievement  (Figure  2). 
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Figure  2.  Influence  on  learner  achievement  when  teachers  practice  instructional  design. 


The  extent  that  classroom  teachers  currently  engage  in  the  kinds  of  activities  practiced  bv  instructional 
designers  is  unclear.  The  basic  assumption  is  that  successful  classroom  teachers  prepare  for  daily 
instruction  in  much  the  same  way  that  instructional  designers  create  or  improve  instruction.  However,  to 
understand  how  gixxi  teachers  express  what  it  is  that  they  "do"  is  essential  to  making  instructional  design 
meaningful  for  them.  It  is  inappropriate  for  the  instructional  design  community  to  change  teacher 
language,  but  it  is  acceptable  to  clarify  and  promote  gixxl  instructional  design  practices."  To  state  specific 
instructional  design  concepts  in  understandable  teacher  language  is  a  vital  ingredient  in  research  on 
insi-uc'ioh.'i  design.  There  is  a  need  to  find  out  if  teachers  understand  the  concepts  of  instructional  design 
when  asked  it  they  "do"  them.  This  calls  for  the  translation  of  instructional  design  practices  into  teacher 
language, 

Wildman  and  Burton  ( 198 1 )  indicated  that  too  much  lime  and  effort  have  been  spent  on  developing 
systematic  approaches  to  the  design  of  instruction  without  knowing  whether  these  approaches  can  have 
widespread  utility  in  the  public  education  sector,  or  whether  the  approaches  have  utility  as  a  device  for 
transforming  theoretical  statements  into  practical  applications.  There  is  evidence  of  pre-serviee  teacher 
mxcr-j  in  acquiring  and  applying  principles  of  learning  and  instructional  design  in  the  public  education 
sector  (Earlc.  1992;  Klein.  1991).  However,  the  reality  of  the  instructional  context  for  public  school 
teachers  may  require  instructional  technologists  to  reconsider  the  value  of  instructional  design  models 
intended  for  applications  in  public  school  environments. 

The  purpose  of  this  study  was  to  collect  and  summarize  data  that  can  be  used  to  correlate  teacher 
planning  practices  and  instructional  designer  practices.  The  following  assumptions  were  made  in  order  to 
formulate  the  research  questions:  (I)  teachers  assume  routines  or  patterns  when  preparing  to  teach,  (2) 
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instructional  designers  practice  mainly  planning  activities,  (3)  the  abundance  of  instructional  design  models 
are  a'  \cd  at  improving  the  instructional  episode,  and  (4)  instructional  design  jargon  adversely  affects  teacher 
perceptions  about  the  value  of  the  systematic  approach.  Based  on  these  assumptions,  the  following 
/  questions  formed  the  impetus  of  this  study;  Is  (here  a  correlation  between  the  planning  routines  of  teachers 
and  the  actions  of  instructional  designers?,  and  What  contextual  factors  affect  the  degree  of  instructional 
design  practiced  by  public  school  teachers? 

Methodology 

Participants 

The  participants  for  this  study  consisted  of  61  public  school  teachers  from  the  northeast  United  States. 
The  participants  included  those  currently  teaching  grades  seven  through  twelve,  and  represented  seven 
subject  areas.  The  participant  group  was  divided  into  two  selected  samples:  1)41  junior  or  senior  high 
school  teachers  from  a  central  New  York  school  district,  and  2)  19  teachers  who  participate  in  a  university- 
school  partnership  program  in  New  York  and  throughout  several  adjacent  states.  Seventeen  junior  high 
school  teachers  were  from  a  single  building,  and  23  high  school  school  teachers  were  from  a  single 
building,  both  in  the  same  school  district.  The  university-school  partnership  teachers  in  addition  to 
teaching  regular  high  school  level  courses  teach  college  level  courses  to  high  school  students  who  receive 
college  credit  for  each  course  successfully  completed. 

The  participants  were  requested  to  complete  a  survey  questionnaire.  One  hundred  ten  surveys  were 
distributed  and  61  returned.  This  yielded  a  return  rate  of  56%.  The  survey  assessed  the  degree  to  which 
teachers  employ  practices  characteristic  of  the  instructional  design  process.  This  was  achieved  through 
combined  efforts  of  interviewing  teachers,  generating  a  list  of  instructional  designer  practices,  and 
translating  the  list  of  instructional  design  practices  into  language  common  to  public  school  teachers. 
Procedures 

A  list  of  instructional  design  practices  was  developed  as  a  result  of  a  content  analysis  based  on  an 
aggregate  of  recommended  design  and  development  competencies  extracted  from  over  60  instructional  design 
models  (Andrews  &  Goodson.  1980:  Branch.  1986:  Briggs,  1977:  Darwa/eh,  1986:  Dick  &  Carey.  1990: 
Gagne,  Briggs  &  Wager.  1992:  International  Board  of  Standards  for  Training.  Performance,  and  Instruction, 
1986:  Kerr,  1981:  Martin,  1984:  Reigeluth,  1983:  Romiszowski.  1981). 

Qualitative  data  about  teacher  planning  routines  was  gathered  to  ascertain  the  language  most  commonly 
used  by  teachers  when  preparing  to  teach.  Open-ended  interviewing  techniques  with  public  school  teachers 
was  used  to  document  "what  goes  through  their  minds"  when  they  think  about  the  teaching  task,  The 
responses  of  15  informants  was  recorded.  An  informal  assessment  of  the  data  indicates  some  natural  overlap 
with  th instructional  designer  practices.  These  data  were  used  to  aid  the  translation  task  in  creating  the 
survey  instrument. 
Instrument 

A  Teacher  Planning  Inventory  (TPI)  served  as  the  survey  instrument.  The  TPI  is  a  35-  item 
questionnaire  divided  into  two  parts,  and  was  used  to  find  out  the  ways  in  which  certain  important  events 
durin?  teacher  planning  are  carried  out  by  teachers.  Part  I  is  comprised  of  the  first  24  items  and  employs  a 
forced-choice  Likert  scale  for  teachers  to  sell-report  their  typical  actions  or  cognitive  processes  while 
preparing  to  teach.  Part  2  of  the  Inventory  requests  demographic  information  about  the  respondent  such  us 
age,  gender,  number  of  years  teaching  and  typical  class  size.  The  list  of  instructional  design  practices  and 
the  Teacher  Planning  Inventory  are  parallel  in  content,  however,  the  language  is  purposefully  different.  The 
Teacher  Planning  Inventory  avoids  instructional  design  jargon  and  uses  language  most  famiiiarto  public 
school  teachers. 

Manv  of  the  things  that  g<x>d  teachers  do,  and  which  are  legitimate  instructional  design  concepts,  are 
done  in  thought  only.  Some  of  the  qualifying  verbs  used  in  the  instructional  designer  priorities  could  be 
revised  to  reflect  the  common  language  of  public  school  teachers.  This  was  addressed  throughout  the 
formation  of  the  inventory,  and  helped  make  the  inventory  more  effective.  Some  of  the  concepts,  such  as 
motivational  tactics  and  formative  evaluation  of  instructional  episodes,  may  be  greeted  by  public  school 
teachers  with  some  skepticism.  Reluctance  to  fully  subscribe  to  the  use  of  instructional  design  models  was 
attributed  more  to  practical  constraints  which  were  viewed  as  "realities"  than  because  teachers  believed  they 
are  unimportant. 
Data  Analysis 

The  overall  mean  recorded  for  each  respondent  on  Part  I  of  the  questionnaire  became  the  score  of  the 
participants.  Frequency  counts  were  conducted  to  identity  relationships  between  participant  scores  by 
categorical  data  reported  in  Part  2,  Crosstabulations  formed  the  primary  data  analysis  to  determine 
correlations. 
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Results 

The  results  indicate  that  a  strong  relationship  exists  between  the  planning  activities  of  teachers  and 
instructional  design  practices  (Table  1).  The  overall  mean  was  7.0  on  a  10  point  scale,  with  a  mean  of  10 
indicating  the  strongest  possible  relationship.  However,  there  appears  to  be  no  correlation  between  years  of 
teaching  experience,  class  size,  grade  level,  and  level  of  teacher  education  and  teacher  practice  of 
instructional  design. 

The  results  from  Table  1  do  suggest  that  the  subject  taught  docs  influence  the  practice  of  instructional 
design  by  teachers.  Science  teachers  reported  the  lowest  mean  of  5.3  compared  a  mean  of  7.7  for  English 
and  Language  teachers.  Given  the  pattern  of  the  mean  scores  on  item  1  through  24  across  the  subject 
matter  groupings  (mathematics,  science,  etc.)  an  analysis  of  variance  was  conducted  (Table  2)  to  reveal  any 
statistically  significant  differences  across  those  groups.  The  results  of  a  liberal  test  of  significance  for 
general  linear  models  indicate  that  subject  taught  affects  the  potential  for  teachers  to  practice  instructional 
design. 

Because  a  large  percentage  of  one  of  the  teacher  groups  (the  University  Partnership  group)  were  math 
teachers,  a  further  test  for  interaction  was  employed.  No  interaction  effect  was  detected  (F=.64,  p=.64) 
indicating  that  subject  matter  taught  could  have  an  effect  on  the  dependent  variable  (Table  3).  However,  due 
to  the  nature  of  the  data  collection  there  is  a  possible  violation  of  the  model,  therefore,  testing  significance 
against  a  more  conservative  critical  value  is  recommended.  Use  of  the  appropriate  Fqq  does  nof indicate 
statistical  significance  of  difference  in  means  across  subject  taught  groups. 

Table  1  Summary  of  Descriptive  Data  for  Contextual  Factors 


Background  Information  n  Mean  S  I) 


Overall 

61 

7.0 

2.5 

Years  of  Teaching  Experience 

56 

7.1 

2.4 

1  -  5 

3 

6.0 

1.7 

6-  10 

9 

6.1 

2.9 

11-15 

8 

6.6 

2.4 

16-20 

19 

7.6 

1.8 

2 1  or  more 

17 

7.3 

2.9 

Class  Size 

58 

7.1 

2.4 

20  or  less 

25 

6.8 

2.6 

21  -35 

33 

7.3 

2.3 

(irade  Level 

2  5 

6.8 

2.8 

Junior  High 

13 

7.0 

2.7 

Senior  High 

6 

5.8 

3.0 

Olhcr 

6 

7.5 

3.3 

Education  Level 

5  7 

7.0 

2.4 

Bachelors 

15 

6.7 

2.3 

Masters 

39 

7.0 

2.4 

Post- Masters 

3 

9.3 

1.2 

Subject  Taught 

5  3 

7.1 

2.4 

Math 

26 

7.6 

2.3 

Science 

7 

5.3 

1.9 

English/  Language 

7 

7.7 

1.9 

Social  Science 

5 

7.6 

2.5 

Other 

8 

6.1 

3.2 
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Table  2  Summary  ANOVA  of  Suhiect  Taught 


DF 

SS 

MS 

F 

P 

Between  Subjects 

4 

6966.30 

1741.58 

3.11 

0.02 

Within  Subjects 

49 

27428.53 

559.77 

Total  Within  Subjects 

53 

34394.83 

Table  3  Interaction  of  Subject  bv  Group 


DF 

SS 

MS 

F 

P 

Subject 

4 

6464.63 

1616.16 

2.69 

.04 

Group 

2 

37.45 

18.73 

.03 

.97 

Subject  into  Group  (Interaction) 

4 

1532.54 

383.13 

.64 

.64 

Conclusion 

The  potential  lor  learner  achievement  is  enhanced  when  teachers  practice  instructional  design.  Based  on 
this  study,  however,  additional  empirical  documentation  is  required.  It  appears  that  some  instructional 
design  practices  may  be  beyond  the  realm  of  manipulation  by  public  school  teachers,  and  therefore  a 
dialogue  between  instructional  design  professionals  and  public  school  teachers  should  be  formalized.  It  is 
reflected  here  that  instructional  designers  should  consider  instructional  design  models  which  combine 
common  teacher  planning  routines  with  instructional  design  practices. 
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The  research  reported  in  this  paper  is  the  result  of  a 
preliminary  investigation  and  development  of  a  knowledge  base  for 
the  field  of  Instructional  Design  and  Technology  (ID&T).  A 
knowledge  base  provides  the  theoretical  and  research 
underpinnings  for  the  field.     The  development  of  this  knowledge 
base  is  important  for  the  education  of  ID&T  specialists  and  the 
definition  of  the  field. 


Description  of  the  Taxonomy 


The  knowledge  base  model  is  adapted  from  the  ID&T  taxonomy 
model  proposed  by  the  Association  for  Educational  Communications 
and  Technology  (AECT)  Definitions  and  Terminology  Committee.  mu 
taxonomy  is  a  three  dimensional  model  as  shown  in  figure  1. 
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Figure  1:  Taxonomy  for  Instructional  Design  and  Technology  as 
proposed  by  the  Association  for  Educational  Comitunications  and 
Technology  Definitions  and  Terminology  Committee,  1991. 
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The  major  side  of  the  model  is  divided  into  four  areas: 
1)  design,  2)  delivery,  3)  evaluation,  and  4)  management.  The 
second  side  divides  each  of  these  four  major  areas  into  subareas 
such  as  instructional  design  and  learning  theory.     The  third 
side  is  divided  into  three  types  of  knowledge  namely:  1) 
research,  2)  theory,  and  3)  philosophy. 

The  purpose  of  this  study  was  to  test  the  feasibility  of 
using  the  model  to  build  an  ID&T  knowledge  base.  The 
feasibility  was  tested  by  mapping  doctoral  dissertations  into  the 
model . 

Background 

Practitioners  in  the  field  of  instructional  technology  are 
faced  with  the  task  of  defining  the  purpose  and  role  of  a  dynamic 
and  emerging  field  of  scientific  thought.    To  complete  this  task 
they  must  establish  a  means  for  organizing  and  identifying  the 
relationships  emerging  from  research,  theory,  and  philosophical 
studies.    Given  the  intricate  and  complex  relationship  among 
factors  such  as  learner  characteristics,  task,  technology,  and 
instructional  design,  the  effort  to  organize  numerous  functional 
categories  of  interest  and  information  can  be  overwhelming. 

The  use  of  taxonomies  and  classifications  to  aid  in 
simplifying  these  relationships  has  been  widely  used  with  small 
knowledge  bases,  especially  in  computer  aided  education  (Carrier 
&  Sales,  1987;  Kozma  &  Bangert-Drowns ,  1987;  Knezek,  Rachlin,  & 
Scannell,  1988).     There  have  not  been  any  attempts  to  develop  a 
taxonomy  that  encompasses  the  entire  field  of  instructional 
technology. 

This  lack  of  a  common  framework  can  lead  to  the  use  of 
"fuzzy"  or  multiple  definitions  for  the  same  idea,  making 
generalizations  across  sub-fields  difficult.    Thus,  the 
development  of  a  taxonomy,  or  common  knowledge  base,  is  important 
to  the  future  development  of  the  field  of  instructional 
technology.     This  paper  addresses  the  validity  and  usefulnesb  of 
a  taxonomic  structure  proposed  by  the  Association  for  Educational 
Communications  and  Technology  (AECT)  Definitions  and  Terminology 
Committee. 

The  origin  of  classification  schemes  or  taxonomies  to 
organize  human  knowledge  bases  dates  back  to  the  Greeks,  and  even 
to  primitive  man  (Sokal,  1974).     The  purpose  of  developing 
classification  schemes  is  to  achieve  economy  of  memory,  and 
ultimately  to  describe  the  structure  ai.d  relationship  of  the 
items  to  each  other  and  to  similar  objects.    This  classification 
structure  aids  in  simplifying  relationships  in  such  a  way  that 
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general  statements  can  be  made  about  classes  of  objects.  (Sokal, 
1974) . 

Fleishman  &  Quaintance  (1984)  identified  several  potential 
benefits  for  developing  a  taxonomy  of  human  performance  that  have 
direct  relevance  to  the  field  of  instructional  technology. 

1.  To  aid  in  conducting  literature  reviews.  Instructional 
technologists  looking  for  literature  on  a  particular  problem 
often  have  difficulty  identifying  appropriate  research 
descriptors  for  their  search  (e.g.  instructional  technology 
versus  educational  technology). 

2.  Create  the  ability  to  generalize  to  new  tasks.     In  the 
case  of  instructional  technology,  the  rapidity  of 
technological  change  necessitates  the  ability  to  transfer 
what  is  known  from  one  technology  to  another.    Without  this 
transferability  the  research  base  must  be  recreated  for  each 
new  technological  innovation. 

3.  Exposing  gaps  in  knowledge  by  delineating  categories  and 
subcategories  on  knowledge,  exposing  holes  in  research  or 
theory . 

4.  Assisting  in  theory  development  by  evaluating  how 
successfully  theory  organizes  the  observational  data 
generated  by  research  within  the  field  of  instructional 
technology. 

Methodology 

The  model  was  tested  by  mapping  ID&T  dissertations  into  the 
three  dimensional  model.     The  dissertation  titles  covered  the 
years  1977  through  1988  and  were  taken  from  existing  lists  of 
ID&T  doctoral  dissertations  (Caffarella  &  Sachs,  1988; 
Caffarella,  1991).     These  lists  contain  1518  dissertations 
completed  at  46  institutions  in  the  United  States. 

A  random  sample  of  152  dissertations  was  drawn  from  these 
lists.     Ecich  of  these  dissertations  was  assigned  to  a  specific 
cell  within  the  model.     This  classification  process  was  completed 
independently  by  two  evaluators.    When  the  classifications 
differed,  the  evaluators  used  consensus  to  determine  the 
appropriate  cell. 

Results 

The  results  of  mapping  the  dissertations  into  the  model  are 
presented  in  tables  1  and  2.     As  shown  in  the  tables,  virtually 
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Table  1 

Number  and  Percentage  of  Dissertation  Titles  in  Each  Cell  for  the 
Research  Plane 


U£j  O  lull 

DELIVERY 

EVALUATION 

MANAGEMENT 

Instructional 

Electronic 

Assessment 

Resource 

Dpq  "i  rm 

T^rhnnl  nrrv 

J.  V-  Vy  1  li  1  W  J.  VM  y 

&  Ana 1 vs i  s 

Manarf  ptnpnt 

A    4-  i  4-1  act  1% 
**     llUlCk)  Jo 

7      W  X  L»  X  ~  O      v  O 

18  titlpc;  12 

Message 

AV  Technology 

Formative 

Project 

Design 

&  Television 

Evaluation 

Management 

5  titles  3% 

13  titles  9% 

0  titles  0% 

4  titles  3% 

Instructional 

Print 

Summative 

Change 

Theory 

Technology 

Evaluation 

Process 

27  titles  18% 

6  titles  4% 

1  titles  1% 

15  titles  10% 

Learning 

General 

Delivery 

Theory 

Delivery 

Management 

30  titles  20% 

1  titles  1% 

3  titles  2% 

Personnel 

Management 

3  titles  2% 

68  titles  45% 

24  titles  17% 

10  titles  7% 

43  titles  29% 

Table  2 

Number  and  Percentage  of  Dissertation  Titles  in  Selected  Cells 
for  the  Theory  and  Philosophy  Planes. 

Instructional  Design  —  Theory 
1  title  1% 

Formative  Evaluation  —  Theory 
1  title  1% 

Instructional  Design  —  Philosophy 
1  title  1% 

Note:  All  other  cells  in  the  theory  and  philosophy  planes 
had  no  title  entries. 

all  of  the  dissertations  were  in  the  research  plane  with  only  two 
dissertations  in  theory  plane  and  one  in  philosophy  plane.  The 
largest  number  of  dissertations  were  in  the  design  plane  with  45% 
of  the  sample.     The  next  largest,  number  was  in  the  management 
column  with  29%  followed  by  delivery  with  16%  and  evaluation  with 
8%. 

The  largest  number  of  dissertations  in  an  individual  cell 
was  learning  theory  at  20%.    This  was  followed  closely  by 
instructional  theory  at  18%.     The  other  cells  with  substantial 
numbers  of  dissertations  were  resource  management  (12%),  change 
process  (10%),  and  AV  technology  (including  television)  (9%). 
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Every  cell  on  the  research  plane  of  the  model,  except 
formative  evaluation,  had  at  least  one  dissertation.     Only  three 
dissertations  were  mapped  into  the  theory  and  philosophy  planes. 
Two  new  cells  were  added  to  the  original  model  in  the  process  of 
mapping  the  titles.     These  cells  are  "General  Delivery"  in  the 
delivery  plane  and  "Personnel  Management"  in  the  management 
plane.     Four  dissertations  (3%  of  the  sample)  did  not  fit  into 
any  of  the  cells  in  the  model. 

To  measure  change  over  time  the  dissertations  were  divided 
into  three  groups  covering  the  years  1977-1980,  1981-1984,  and 
1985-1988.     The  percentage  of  dissertations  in  each  cell  for 
1977-1980  were  compared  against  the  percentages  for  1985-1988. 
The  percentages ,  as  shown  in  table  3 ,  were  relatively  equal  for 
the  major  categories  identified  abo^e.     The  only  cell  with  a 
major  increase  was  electronic  technology  which  rose  from  no 
dissertations  during  the  1977-1980  period  to  9%  of  the  total  for 
the  1985-1988  period. 

Table  3 

Percentage  of  Dissertations  in  Major  Categories  for  the 
Periods  1977-1980  and  1985-1988 


Category  1977-1980  1985-1988 

Percentage  Percentage 

Instructional  Theory  19%  18% 

Learning  Theory  26%  23% 

Electronic  Technology  0%  9% 

AV  Technology  5%  7% 

Resource  Management  7%  11% 

Change  Process  12%  11% 


Conclusions 

There  are  several  conclusions  from  this  investigation. 
First,  the  model  is  a  reasonable  representation  of  the  ID&T 
knowledge  base.    Ninety-seven  percent  of  the  dissertations  fit 
into  an  appropriate  cell  on  the  model  with  only  four 
dissertations  not  fitting  the  model.     Therefore,  the  model  does 
reflect  the  doctoral  dissertation  research  that  is  being 
conducted  in  the  United  States. 

The  next  conclusion  is  that  the  ID&T  dissertation  research 
has  a  strong  design  element.     Forty-five  percent  of  the 
dissertations  were  in  the  design  plane  of  the  model  with  18%  in 
instructional  theory  and  20%  in  learning  theory.    A  relatively 
small  percentage  (16%)  of  the  dissertations  dealt  with  the 
delivery  of  instruction  and  the  hardware  components  of  the  field. 
Thus  research  in  the  field  is  not  largely  hardware  oriented,  as 
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many  outsiders  believe,  but  has  strong  research  grounding  in  the 
design  of  instruction,  particularly  instructional  and  learning 
theory . 

A  substantial  number  (10%)  of  the  dissertations  dealt  with 
the  change  process.     This  is  not  a  topic  traditionally  associated 
with  the  ID&T  field  but  is  a  major  factor  in  the  successful 
introduction  of  new  technologies  into  educational  and  training 
settings.     This  will  likely  evolve  as  an  important  component  of 
the  field  in  the  future. 

Virtually  all  of  the  dissertations  were  classified  as 
research  with  only  2%  being  classified  as  either  theory  or 
philosophy.     This  finding  is  not  surprising  since  theory  building 
and  philosophical  investigations  are  usually  conducted  by 
experienced  scholars  in  the  field.     Since  this  study  used 
doctoral  dissertations  there  were  not  any  experienced  scholars  in 
the  sample. 

The  last  conclusion  parallels  the  development  and 
availability  of  microcomputers  over  the  past  dozen  years.  Prior 
to  1980  there  were  not  any  dissertations  dealing  with  electronic 
technology  systems.     During  the  period  1985-1988  dissertations  of 
this  topic  represented  9%  of  the  total . 

Limitations 

There  are  several  limitations  to  the  investigation  in  this 
study.     The  dissertations  were  assigned  to  the  various  cells 
based  upon  the  title  of  the  dissertation.    Although  dissertation 
titles  tend  to  be  very  descriptive,  many  failed  to  identify  the 
variables  under  investigation.     The  meaning  of  the  specific  terms 
in  the  titles  is  also  subject  to  interpretation.     Therefore,  the 
use  of  titles  only  may  have  lead  to  the  assignment  of  some 
dissertations  to  improper  cells. 

The  boundaries  between  the  cells  are  not  absolute 
boundaries  and  some  dissertations  could  be  classified  in  multiple 
cells.     For  example,  the  boundary  between  instructional  theory 
and  learning  theory  is  not  an  absolute  point  and  some  studies 
could  logically  be  classified  in  either  or  both  cells. 

Suggestions  for  Further  Investigation 

The  model  should  be  tested  with  other  scholarly  literature 
from  the  field  of  ID&T .     Specifically  the  articles  from  prominent 
journals  should  be  mapped  to  test  the  validity  of  the  model  and 
to  better  define  the  knowledge  base  for  the  field.     If  this 
mapping  proves  equally  valid  then  the  model  should  be  used  to 
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Developing  a  Knowledge  Base  and  Taxonomy 

7 

build  a  complete  knowledge  base  for  the  field  of  instructional 
design  and  technology. 
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The  effects  of  computer-assisted  learning-strategy 
training  on  the  achievement  of  learning  objectives 

Introduction 

Teaching  individuals  learning  strategies  is f  in 
effect,   teaching  someone  how  to  learn.     If  learning  is  a 
knowledge  domain  in  the  same  way  as  any  subject,  for 
example  chemistry  or  history,  then  successful  learners 
have  organized  knowledge  about  learning  processes  in  the 
same  way  that,  chemists  or  historians  have  organized 
knowledge  for  their  domains  (E.     Gagne,  1985).     There  is 
a  real  possibility  of  improving  and  enhancing  learning 
abilities  through  special  programs  of  training, 
education,  and  other  types  of  intervention  (Carroll, 
1989).     This  study  examined  the  effects  of  computer- 
assisted  learning-strategy  training  on  learning  from  a 
computer-based  tutorial  about  microcomputer  components. 
Previous  Research 

Snowman  (1986)  describes  learning  tactics  as 
specific  techniques  used  to  successfully  complete  a 
learning  task  while  strategies  are  general  plans  decided 
upon  before  engaging  in  a  learning  activity.  He 
discriminates  among  kinds  of  tactics  by  their  objectives 
which  may  be  memory-directed  or  comprehension-directed. 
Memory-directed  tactics  include  the  use  of  mnemonics, 
outlining,  underlining,  and  summarizing.     He  refers  to 
questioning  as  a  "comprehension-directed  tactic  geared 
more  toward  understanding  the  meaning  of  ideas  and  their 
inter-relationships  than  toward  ensuring  reliable  recall 
of  those  ideas"   (p,  256). 
Generating  Questions 

Levin  (1982)  discriminated  between  the  types  of 
generated  questions  and  their  effect  on  types  of  test 
questions.     Duell  (1978)  found  that  college  students  who 
generated  knowledge  and  application  level  questions 
scored  significantly  better  than  students  who  did  not 
generate  questions  and  that  learners  generated  low-level 
knowledge  questions  which  were  not  effective  for  high- 
level  tests.     Andre  and  Anderson  (1979)  found  that 
learner-generated  questions  were  more  effective  than  re- 
reading.    They  also  found  that  answering  teacher- 
generated  questions  were  equally  effective.     Singer  and 
Dcnlan  (1982)  found  that  students  who  were  trained 
benefitted  more  from  student-generated  questions  than 
those  generated  by  the  teacher. 

Training  students  to  generate  questions  has  been  an 
issue  in  many  studies,  but  researchers  observed  mixed 
results  from  variables  such  as  length  and  intensity  of 
training,  presence  or  absence  of  metacogni tive 
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components ,  and  varied  knowledge  levels  (Andre  & 
Anderson,  1978-1979;  Dreher  &  Gambrell,  1982;  Singer  & 
Donlan,  1982;  Frase  &  Schwartz,  1975). 

Simpson  (1986)  investigated  the  use  of  writing  as  an 
independent  learning  strategy  and  developed  PORPE 
(Predict ,  Organize,  Rehearse,  Practice,  Evaluate) .  This 
strategy  was  designed  to  prepare  students  for  success 
with  essay  exams.     The  "Predict"  phase  of  PORPE  asked 
studentn  to  generate  potential  essay  questions  after 
reading.     Predicted  essay  questions  were  directed  it 
higher  levels  of  knowledge.     Students  were  instructed  to 
use  verbs  such  as  explain,  criticize,  compare  and 
contrast .     Simpson  (  1986 )  found  that  the  strategy 
required  time  to  master  but  was  effective. 
Anthony  and  Raphael   (1987)  examined  the  effects  of 
questioning  strategies  and  found  that  when  subjects 
generated  low-level  questions,   the  strategy  was 
ineffective •     Thus,   it  is  apparent  that  students  must  be 
taught  how  to  generate  effective  questions  d'^ected 
towards  the  level  of  learning  to  be  tested. 

Purpose  of  study 

Many  educational  psychologists  agree  that  students 
need  to  learn  strategies  (Kulik,  Kulik,   &  Schwab,  1983), 
However,  the  results  of  training  programs  vary  as  much  as 
the  training  formats  and  delivery  systems  (Armbruster, 
1987;  Cook  &  Mayer,   1988;  Holley  &  Donsereau,   1984),  A 
systematic  evaluation  of  strategy- training  CAI  would 
contribute  to  the  field  of  study  and  provide  prototypical 
software.     Since  microcomputers  provide  a  learning 
envi ronment  that  favors  individual ization,  sel f -pacing , 
practice,  and  immediate  feedback  (Hannafin  &  Peck,  1988), 
it  is  reasonable  to  consider  computer -del ivered  learning- 
strategy  training.     The  goal  of  this  study  was  to 
determine  the  effectiveness  of  specific  strategy  training 
on  learning  about  microcomputer  components  with  CAI 
assessed  by  specific  levels  of  test  objectives. 

Information  Processing  and  Memory-directed  Strategies 

Mayer  proposed  four  components  of  the  encoding 
process:  selection,  acquisition,  construction,  and 
integration  (Mayer,   1988;    Mayor  &  Greeno,  1972; 
Weinstein  &  Mayer,  1986).     During  selection,   the  learner 
attends  to  information  received  by  the  sensory  register. 
Acquisition  provides  for  active  transfer  of  information 
from  working  memory  to  long-term  memory.  During 
construction,   the  learner  connects  ideas  and  develops  an 
outline  organization  or  schema  (Bransford,   1979)  or, 
while  reading,  forms  a  macrost ructure  (Kintsch  and  van 
Dijk,  1978).     Finally,   integration  of  newly  acquired 


o 

ERIC 


105  125 


information  with  information  already  held  in  long-term 
memory  ia  necessary  for  learning  transfer.     To  he 
effectives,  memory-directed  learning-strategy  training 
must  address  these  mental  processes. 

Levels  of  Learning  and  Comprehension-directed  Strategies 

Intellectual  skills  can  be  categorized  by  complexity 
trom  skills  requiring  discrimination,  conceptualizations, 
applying  rules,  and  finally,  problem  solving  (Gagne', 
Briggs,  and  Wager,  1988).     Teaching  learners  about  these 
levels  of  complexity  and  applying  that  information  to 
generating  questions  can,   in  and  of  itself,  be  used  as  a 
form  of  learning-strategy  training.     By  generating 
questions  about  information,  such  as  microcomputer 
components,   learners  should  understand  the  content  well 
enough  to  apply  rules  and  solve  problems.  Delivering 
that  training  by  microcomputer-based  tutorials  should 
facilitate  learning  because  of  the  characteristics  of 
CAI. 

Method 

Subjects 

Subjects  consisted  of  60  undergraduate  students 
enrolled  in  introductory  computing  courses  at  Old 
Dominion  University  in  southeastern  Virginia.     The  mean 
age  of  all  students  was  28  years  and  ranged  from  19  to  43 
y°ars.     The  subjects  included  only  those  students  whose 
prior  experience  with  computers  consisted  of  very  little 
word  processing,  playing  games,  or  using  computerized 
cash  registers.     As  a  result  of  a  survey  administered  to 
80  students,  those  students  who  reported  taking  prior 
computing  classes  were  excluded  from  the  study.     In  this 
balanced  design,  subjects  were  randomly  assigned  to  one 
of  three  treatment  groups  consisting  of  comprehension- 
directed  strategy  tutorials,  memory-directed  strategy 
tutorials,  or  a  control  group.     As  computing  students, 
subjects  received  5  participation  points  towards  their 
final  grade. 

Instructional  and  Assessment  Materials 

A  tutorial  was  developed  in  order  to  introduce  the 
learner  to  microcomputer  components.     The  tutorial  was 
created  using  Linkway,  an  authoring  system.      The  learner 
had  the  option  of  selecting  topics  by  clicking  the  mouse 
on  t-he  approriate  space  located  on  a  graphic 
representation  of  a  microcomputer.     Learners  accessed  the 
tutorial  in  a  Novell  LAN  microcomputer  laboratory.  The 
tutorial  covered  topics  included  under  the  headings: 
central  processing  unit,  memory,  secondary  storage,  input 
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and  output  devices,  and  peripherals.     Wherever  possible, 
graphic  representations  of  components  supported  verbal 
descriptions  and  conceptualizations. 

Two  learning  strategy  tutorials,  developed  through 
the  use  of  Linkway,  addressed  comprehension-directed 
strategies  or  memory-directed  strategies.     Both  tutorials 
addressed  the  same  content,  microcomputer  components,  and 
were  erected  within  the  same  format.     The  same  colors, 
navigational  buttons,  and  fonts  were  used  for  each  screen 
and  both  tutorials  possessed  the  same  number  of  screens 
or  Linkway  Pages.     Both  tutorials  concluded  with  the  same 
practice  quiz  about  microcomputer  components.  Each 
tutorial  began  with  the  same  Linkway  Folder,  analogous  to 
a  HyperCard  Stack,   that  discussed  Mayer's  information 
processing  model  accompanied  by  examples  and 
opportunities  for  interaction.     Each  of  Mayer's 
components,   including  selective  attention,  acquisition, 
construction,  and  integration,  was  discussed  and  examples 
provided . 

The  tutorial  that  addressed  memory-directed 
strategies  explained  tactics  to  enhance  memory  including 
outlining,  organizing  information  into  graphic 
organizers,  and  linking  new  information  with  prior 
knowledge.     Learners  were  directed  to  apply  those  tactics 
to  the  information  they  learned  from  the  microcomputer- 
components  tutorial  by  creating  outlines,  graphic 
organizers  and  relating  new  information  to  prior 
knowledge  using  the  notes  Miey  took  from  the 
microcomputer-components  tutorial.     The  student's  name, 
age,  responses  to  input  questions,  and  generated 
materials  were  written  to  data  files  on  disk.  These 
files  were  available  for  inspection  by  the  investigator. 

The  comprehension-directed  tutorial  elaborated  on 
the  levels  of  learning  and  generating  questions  according 
to  the  levels  of  learning.     The  tutorial  required  that 
students  create  questions  at  each  level  of  learning  by 
typing  into  designated  fields  within  the  tutorial. 
Sample  questions  were  provided  and  students  were 
permitted  to  access  their  notes  from  the  microcomputer- 
component  tutorial.     Due  to  the  nature  of  Linkway, 
questions  that  the  students  generated  and  entered  into 
the  fields  were  automatically  saved  when  they  exited  che 
program  so  that  the  investigator  touid  review  results. 
In  addition,   the  student's  name,  age,  and  responses  to 
input  questions  were  written  to  data  files  on  disk. 
Evaluation  Instruments 

Subjects  completed  two,  delayed,  and  separately 
administered  tests:  one  requiring  cued  recall  and  the 
other  requiring  recognition.     The  cued-recall  test  listed 
50  terms  which  included  microcomputer  components, 
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information  about  memory,  and  computing  acronyms. 
Students  were  directed  to  provide  definitions  and/or 
descriptions  of  each  term.     The  terms  were  derived  from 
the  microcomputer  tutorial.     The  recognition  test 
included  50  multiple-choice  items  with  10  items  at  each 
learning  level  including  di scr iminat i ons ,  concrete 
concepts,  defined  concepts,   rules,  and  problem  solving. 
The  tests  were  validated  with  the  assistance  of  computing 
instructors. 

Procedures 

This  study  extended  over  the  period  of  one  week.  On 
the  first  day  of  the  study,   students  completed  the 
microcomputer-components  tutorial  in  75  minutes.  The 
microcomputer-components  tutorial  directed  learners  to 
take  notes  on  the  information  but  did  not  provide  any 
specific  note-taking  strategies.     Two  days  later, 
treatment  groups  completed  the  learning  strategy 
tutorials  in  75  minutes.     On  the  same  day,   the  control 
group  continued  to  review  the  microcomputer-components 
tutorial  without  learning-strategy  training  for  75 
minutes.     One  week  later,  students  completed  the  recall 
test  in  25  minutes  and  submitted  it  to  the  investigator. 
Upon  completion  of  the  cued-recall  test,  students 
completed  the  recognition  test  in  45  minutes.  In 
addition,  all  data  files  saved  to  disk  were  reviewed  by 
the  investigator  to  confirm  tutorial  completion  along 
with  inspection  of  the  fields  where  the  learners  typed 
their  questions  or  outlines. 

Resul ts 

Cued-recal 1  Test 

As  seen  in  Table  1,   the  group  mean  scores  for  the 
cued-recall  test  were  at  or  below  50%  of  the  total 
points.     The  analysis  of  variance,  however,  indicates 
that  the  resulta  were  significant  at  the  p  <   .001  level 
with  an  F(2,   57)  =  11.10  due  to  the  learning  strategy 
training.     Comparing  means  using  the  method  of  Least 
Significant  Difference  (LSD)   indicated  that  the  treatment 
group  mean  scores  were  signficantly  higher  than  the 
control  group  mean  scores.     However,  the  comprehension- 
directed  strategy  training  did  not  have  a  significantly 
different  effect  on  learning  as  compared  to  the  memory- 
directed  strategy  training  on  the  cued-recall  test. 

Recognition  Test 

The  group  mean  scores  from  the  recognition  test, 
found  in  Table  1,  ranged  from  33.8  to  43.8  out  of  a  total 
of  50  possible  items.     According  to  an  item  analysis, 
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approximately  60%  of  the  subjects  incorrectly  responded 
to  questions  related  to  computer  memory.     The  ANOVA 
indicates  that  the  main  effects  were  significant  at  the  p 
<  0.001  level  with  F(2,57)  =  13.31.     A  comparison  of 
means  using  the  LSD  method  indicated  that  the  group  mean 
scores  from  subjects  receiving  comprehension-di rec ted 
strategy  training  was  significantly  higher  than  the  group 
mean  scores  from  subjects  receiving  the  memory-directed 
strategy  training  at  the  p  <0.05  level.     Both  treatment 
groups  scored  significantly  higher  than  the  control  group 
on  the  recognition  test. 

Discussion 

The  results  of  this  study  supported  the  hypothesis 
that  microcomputer-delivered  comprehension-directed 
learning  strategy  training  enhances  learning  processes 
assessed  at  the  same  levels  of  learning  as  the  evaluation 
instrument.     Since  text-based  and  lecture-oriented 
learning  strategy  training  has  been  shown  to  be  effective 
(Kulik,  et  al.,  1983),   this  result  is  not  surprising  but 
adds  to  an  increasing  body  of  research  into  the  effects 
of  microcomputer-based  training  (Foreman,   1990;  Jensen, 
1991;  Laridon,   1990)  . 

The  generally  low  mean  scores  from  the  cued-recall 
test  for  both  treatment  groups  and  the  control  group 
suggest  retrieval  processes  were  not  enhanced.     This  may 
have  been  due  to  insufficient  interaction  with  the 
material  for  activation  of  memory  that  would  otherwise 
result  from  greater  depth  of  processing  (Craik  & 
Lockhart,  1972;  Craik  &  Tulving,  1975).     In  spite  of  the 
low  mean  scores,  however,  students  who  received  the 
learning-strategy  training  recalled  significantly  more 
terms  than  students  who  studied  the  microcomputer- 
components  tutorial  ev*n  though  the  control  group 
interacted  more  with  the  informational  materials  than  the 
treatment  groups.     This  supports  the  findings  of  other 
research  conclusions  Lhat  teaching  individuals  how  to 
learn  enhances  learning  processes  (Armbruster,  1987;  Cook 
&  Mayer,  1988;  Dufflemeyer,  1987).     Since  students  who 
did  not  receive  learning-strategy  training  scored 
significantly  lower  than  the  students  in  the  treatment 
groups,   the  study  methods  that  they  chose  were  less 
effective  than  those  learned  by  the  treatment  groups 
(Wade  &  Trather,  1989). 

The  results  from  the  recognition  test  indicate  that 
student-generated  questions  significantly  enhanced  the 
recognition  of  correct  answers  using  discrimination, 
conceptualization,  rule  application,  and  problem  solving 
cn  a  multiple-choice  test.     This  supports  previous 
research  about  the  use  of  learner-generated  questions 
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(Andre  &  Anderson,   1978-1979;  Dreher  &  Gambrell,  1982; 
Duell,   1978;  Levin,   1982;  Simpson,  1986;  Singer  &  Donlan, 
1982;  Frase  &  Schwartz,  1975).     Since  the  level  of 
knowledge  for  the  test  objectives  matched  the  level  of 
questions  generated  by  students,  the  tactic  was 
effective.     This  supports  the  findings  of  Levin  (1982) 
and  Duell   (1978)  who  found  that  the  role  of  consistency 
and  congruency  in  testing  is  critical  to  the  selection 
and  implementation  of  learning  strategies  and  tactics. 

The  effects  of  the  memory-directed  learning  strategy 
tutorial  were  more  significant  than  no  strategy  training 
and  supports  Weinstein's  conclusions  that  some  strategy 
use  is  more  effective  than  non-use  (Weinstein,  1982).  In 
addition,   item  analysis  suggests  that  memory-directed 
strategies  supported  learning  at  the  lower  levels  of 
discrimination  and  concrete  concepts.     This  suggests  that 
in  cases  where  memorization  of  basic  vocabulary  is 
critical,  memory-directed  tactics  will  enhance  learning. 
Rule-using  and  problem  solving  were  enhanced  by 
comprehension-directed  tactics  and  strategies.  This 
supports  Snowman's  (1986)  statement  that  questioning  is  a 
"comprehension-directed  tactic  geared  more  toward 
understanding  the  r  aaning  of  ideas  and  their  inter- 
relationships than  toward  ensuring  re 1 i able  re call  of 
those  ideas"   (p.   256).     This  also  explains  th^  lack  of 
effectiveness  on  retrieval  processes. 

Impl ications 

The  findings  of  this  study  support  the  use  of  CAI 
for  learning-strategy  training.     While  the  learning 
tactics  and  strategies  were  presented  separately  in  this 
study,   further  research  into  the  effects  of  embedded 
strategies  would  conti ibute  to  an  increasing  body  of 
knowledge.     The  consiatency  and  congruency  of  test  items 
with  strategy  training  and  specific  content  areas  would 
also  provide  a  potential  area  for  research.  Additional 
study  about  the  effects  of  various  CAI  characteristics 
and  computer-baced  learning-strategy  training  offers  a 
tremendous  potential  for  investigation. 
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Table  1.     Group  Mean  Scores  for  the  Cued  Recall  and 
Recognition  Tests. 


CUED  RECALL 


Standard 
Mean  Deviation 
Comprehension-directed  Training  26.3  9.889 

Memory-directed  Training  25.9  10.612 

Control  Group  13.8  7.891 


RECOGNITION 


Standard 
Mean  Deviation 
Comprehension-directed  Training  43.8  3.778 

Memory-directed  Training  39.4  6.021 

Control  Group  33.8  7.918 
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Introduction 

The  development  of  constructs  that  underlie 
learning,  particularly  academic  learning  has  always  been 
a  hotbed  for  inquiry,  in  as  much  as  we  humans  are 
characterized  by  innate  curiosity  and  the  capacity  \:o 
learn  and  to  understand  what  we  are  learning.  However, 
while  this  has  primarily  been  the  domain  of  philosophers, 
the  emergence  of  cognitive  psychology  as  a  research 
discipline  has  provided  us  with  more  of  the  pieces  needed 
to  understand  learning  in  context  of  cognitive  processes, 
internal  influences  (i.e.,  individual  characteristics) 
and  external  influences  (i.e.,  sociocultural  influences)  . 

More  specifically,  the  area  of  metacognition  has 
played  a  critical  role  in  understanding  what  develops 
when  an  individual  learns  within  academic  settings  like 
those  generally  found  in  the  United  States  today.  One  of 
the  manifestations  of  metacognitive  thought  is  self- 
assessment.  Broadly  defined,  self -assessment  is  the  act 
of evaluating  or  monitoring  one's  own  level  of  knowledge, 
performance,  and  understanding. 

In  this  paper,  we  will  attempt  to  discuss  self- 
assessment  within  a  metacognitive  framework  and  take  into 
account  the  contexts  in  which  self -assessment  occurs. 
Towards  this  end,  we  will  first  give  a  brief  discussion 
of  metacognitive  and  social  psychology  theories  that 
pertain  to  self -assessment .  Vie  will  then  offer  an 
interactive  model  of  self-assessment  which  delineates 
some  of  the  possible  influences  on  self-assessment 
behaviors.  Finally,  we  will  discuss  the  research 
implications  of  such  a  model  on  approaches  to  studying 
self -assessment . 

Theoretical  Orientations 

The  Metacognitive  Literature 

Metacognition  may  be  generally  defined  as  any 
cognitive  activity  that  concerns  itself  with  any  aspect 
of  any  cognitive  enterprise  (Flavell,  1979) . 
Metacognition  is  believed  to  be  active  in  such  activities 
as  communication,  reading  comprehension,  perception, 
problem  solving,  social  cognition  and  varying  forms  of 
self-instruction.  'tfe  believe  the  work  of  Brown  and 
DeLoache  (1978)  and  the  work  of  Flavell  (1979)  are 
particularly  pertinent  and  instructive  in  positing  a 
model  for  self-assessment. 
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Brown  and  DaLoache  (1978).  They  assert  that  one  of 
the  most  fundamental  differences  between  the  experienced 
learner  and  the  naive  learner  is  the  increasing  capacity 
for  metacognition,  which  they  call  self "regulation  and 
control  of  knowledge.  They  believe  that  along  with  an 
increase  in  learning,  there  is  a  concurrent  increase  in 
metacognitive  activity,  which  allows  the  individual  to 
monitor  problem  solving  techniques  and  then  modify  them. 

Brown  and  DeLoache  further  state  that  individuals 
follow  a  relatively  similar  pattern  of  learning.  First 
the  absolute  novice  shows  little  self-regulation 
(metacognitive  activity) .  Then,  as  he  becomes  more 
familiar  with  the  necessary  rules  and  subprocesses 
required  to  learn  the  subject,  he  increasingly  enters  a 
more  active  period  of  deliberate  self -regulation. 
Finally,  there  is  a  period  when  the  use  of  rules  and 
subprocesses  to  access  knowledge  become  mostly 
automatic.  The  individual  has  becoma  an  expert.  Though 
their  studies  have  primarily  been  on  children,  we  believe 
that  at  any  age,  this  developmental  pattern  is  operative. 
What  develops  seems  to  be  a  schema  or  internalized 
working  model  of  t*e  individual's  self-regulation  or,  in 
our  terms,  the  individual's  self -assessment  process. 

Although  we  may  intuitively  believe  that  adults  have 
a  sophisticated  and  fully  developed  schema  of  s»lf- 
regulation  or  self -assessment,  it  may  not  necessarily  be 
the  case.  What  may  distinguish  the  acquisition  of  this 
schema  may  be  the  constraints  of  internal  influences 
(individual  characteristics)  and  external  influences 
(sociocultural  influences) . 

Flavell  i  1979) .  One  of  the  more  renowned  theorists 
in  the  metacognitive  area,  Flavell  views  metacognition  in 
terms  of  metacognitive  knowledge  and  metacognitive 
experience. 

Metacognitive  knowledge  denotes  the  area  of  acquired 
knowledge  ti^at  deals  with  what  an  individual  knows  about 
how  .ie  thinks  and  how  others  think.  That  is,  it  is  the 
knowledge  and  beliefs  that  one  has  accumulated  through 
experience  which  concern  the  human  mind  and  its  cognitive 
processes  Some  of  this  stored  knowledge  is  declarative 
(e.g.,  knowing  that  you  have  poor  memory).  Other 
metacognitive  knowledge  may  be  procedural  (e.g., you  know 
how  to  improve  your  poor  memory  by  the  use  of 
supplementary  aids  like  notes  or  lists)  .There  is  also 
metacognitive  knowledge  that  is  both  procedural  and 
declarative  (e.g.,  your  ability  to  know  that  writing 
lists  helps  your  poor  memory) . 

Flavell  views  metacognitive  knowledge  as  roughly 
subdivided  into  knowledge  about  persons,  tasks,  and 
strategies. 

Persons  includes  knowledge  and  beliefs  one  might 
acquire  concerning  what  individuals  are  like  as  thinkers. 
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It  can  be  further  broken  down  into  knowledge  and  beliefs 
about  cognitive  differences  within  people  (e.g. ,  you  know 
that  you  are  better  at  one  subject  than  another)  ,  between 
people  (e.g. , knowledge  about  other  people's  social 
cognitive  skills) ,  and  cognitive  similarities  among  all 
people  (in  other  words,  the  universal  properties  of  human 
cognition. 

Tasks  includes  two  subcategories.  First  is  the 
nature  of  the  information  to  be  addressed  when  facing  a 
cognitive  task  (e.g.,  the  difficulty  of  taking  a 
comprehensive  exam) .  Second  is  the  nature  of  the  task 
demands  (e.g.,  You  know  it  is  easier  to  remember  the 
general  theme  of  a  story  than  its  exact  wording) . 

Strategies  includes  your  knowledge  of  the  ways  that 
you  can  succeed  in  achieving  your  cognitive  goals  (e.g., 
remembering  a  poem,  comprehending  a  theory,  or  solving  a 
problem)  .  The  principal  role  of  metacognitive  strategies 
is  to  help  you  succ  jssfully  carry  out  cognitive 
activities  (e.g.  ,  knowing  that  you  must  monitor  your  time 
during  an  exam) . 

Finally,  Flavell  believes  that  metacognitive 
knowledge  is  not  qualitatively  different  from  other  kinds 
of  knowledge.  Like  other  knowledge  it  is  probably 
accumulated  in  a  slow  and  gradual  fashion  through 
experiences  in  various  cognitive  activities.  It  is 
probably  often  automatically  activated  through  stimulus 
response  processes  that  detect  and  appropriately  respond 
to  familiar  cognitive  situations.  Also,  metacognitive 
knowledge  can  oftentimes  be  insufficient,  inaccurate,  not 
reliably  retrieved  and  not  used  when  appropriate. 

Metacognitive  experience  is  a  concept  which  refers 
to  cognitive  or  affective  experiences  that  occur  during 
some  activity  that  gives  insight  to  that  activity. 
Metacognitive  experiences  can  be  brief  and  simple  (e.g. , 
a  brief  puzzlement  about  a  friend's  behavior)  or  lengthy 
and  comeple  (an  obssessive  desire  to  understand  a 
friend's  behavior).  Thny  may  also  occur  at  any  time 
before,  during,  or  after  a  thinking  activity  (e.g., 
thinking  about  how  you  are  doing  on  different  parts  of  an 
exam) .  Many  metacognitive  experiences  tend  to  include 
your  perpception  of  previous  progress  you  have  made, 
progress  you  are  currently  making,  or  progress  you  will 
make  in  a  given  activity.  Such  metacognitive  experiences 
occur  in  situations  that  would  be  expected  to  engender 
careful,  conscious  monitoring  and  regulation  of  one's  own 
cognition.  Metacognitive  experiences  serve  useful 
functions  in  ongoing  cognitive  activities.  For  instance, 
if  you  have  trouble  solving  a  problem,  a  metacognitive 
experience  may  help  you  to  restructure  your  thinking  so 
that  you  can  reach  a  solution.  In  this  sense,  a 
metacognitive  experience  is  adaptive.  Moreover, 
metacognitive    experiences   tend   to   be    influenced  and 
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shaped  by  whatever  relevant  metaccqnitive  knowledge  you 
have  acquired.  In  turn,  metacognitive  experiences  must 
also  contribute  information  about  persons,  tasks,  and 
strategies  to  one's  developing  store  of  metacognitive 
knowledge. 

According  to  Flavell,  it  would  seem  that 
metacognitive  knowledge,  metacognitive  experience,  and 
cognitive  behavior  are  constantly  informing  and  eliciting 
one  another  during  the  course  of  a  task. 

Social  Psychology  Literature 

Learning  does  not  occur  in  a  vacuum  and  is 
influenced  by  a  number  of  different  components  which  can 
be  loosely  grouped  into  external  influences 
(sociocultural  influences)  and  internal  influences 
(individual  characteristics)  .  The  work  of  social 
cognitivists  and  the  emerging  field  of  cultural 
psychology  is  particular  salient  to  our  proposed  model 
for  self -assessment. 

Social  cognitivists  like  Vygotsky  (1962)  and  Cole  and 
Scribner  (1975)  bring  to  cognitive  psychology  the  premise 
that  cognitive  activities  develop  and  are  modified  within 
sociohistorical  and  cultural  contexts.  The  effect  of 
these  influences  can  be  observed  in  an  individual's 
choice  of  activities,  motives,  and  priorities.  Thus,  the 
individual's  schema  for  self-assessment  is  based  not  only 
on  metacognitive  knowledge  and  experience  but  also  on  a 
variety  of  internal  and  external  influences.  The 
emerging  discipline  of  cultural  psychology  also  speaks  to 
this  issue  in  that  it  views  individual  development  and 
functioning  as  occurring  in  particular  intentional 
worlds — that  is  within  unique  sociocultural  environments 
that  have  within  them  unique  communities  of  individuals 
with  unique  beliefs,  desires,  emotions  and  purposes 
(Shweder,  1990) .  From  this  perspective,  we  would  expect 
metacognition  to  be  influenced  by  an  individual's  level 
of  involvement  and  commitment  along  different  social 
domains  (i.e. , school,  family,  workplace,  etc.).  In  this 
way,  learners  (at  any  age)  can  be  seen  as  belonging  to 
interconnected  systems  which  influence  their  learning  at 
every  turn. 
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Possible  Model  for  Self-Assessment 

What  higher  order  thinking  occurs  during  an 
individual's  self-assessment?  What  influences  this  self- 
assessment  capacity?  How  can  we  improve  an  individual's 
ability  to  self-assess,  and  does  this  ability  hold  along 
different  knowledge  domains?  We  believe  these  questions 
can  best  be  answered  by  positing  a  self-assessment  model 
that  takes  into  account  the  metacognitive  and  social 
psychology  theories  we  discussed  above. 

In  this  model,  self -assessment  behaviors  are  really 
manifestations  of  metacognitive  processes  involving 
metacognitive  knowledge  and  metacognitive  experience. 
These  metacognitive  processes  make  up  an  individual's 
schema  for  self -assessment.  This  schema  is  part  of  the 
individual's  overall  cognitive  processes.  However,  at 
the  same  time,  the  self-assesment  schema  acts  as  a 
mediator  of  cognitive  activities  in  different  knowledge 
domains.  Thus,  we  may  view  the  self -assessment  schema 
as  a  generalized  cognitive  process  that  is  operable  along 
a  wide  array  of  knowledge  domains. 

The  operation  of  this  self-assessment  schema  may 
occur  at  two  levels.  During  a  task,  a  person  may  perform 
frequent  incremental  assessments  of  his  performance  in 
order  to  reach  completion.  The  self -assessment  schema  is 
utilized  in  a  formative  fashion  in  order  to  monitor  and 
modify  the  on-going  task.  Once  the  individual  has 
completed  the  task,  he  again  utilizes  the  self-assessment 
schema  to  do  a  summative  evaluation  of  his  performance  on 
the  overall  task.  He  may  then  use  this  summative 
evaluation  to  modify  his  self -assessment  schema,  which 
will  be  used  again  at  some  future  task. 

However,  all  cognitive  processes  (e.g., 
metacognition  and  particularly  self -assessment)  do  not 
merely  occur  independent  of  other  influences,  as  we 
mentioned  earlier  in  our  social  psychology  section. 

External  influences  in  an  academic  environment  may 
include  the  school.  Within  a  school  setting,  the  goals, 
curriculum,  training,  faculty-student  interaction  ,  etc. 
may  affect  the  individual's  ability  to  self -assess.  The 
individual's  family  life,  culture,  neighborhood,  and  the 
media  may  also  impact  on  self -assessment . 

Any  task  which  requires  the  use  of  the  self- 
assessment  schema  is  itself  embedded  within  these 
external  influences.  The  tasks  may  come  from  the  school 
(e.g.,  performance  on  exams),  from  the  family  (e.g., 
interaction  dynamics  among  family  members) ,  or  from  the 
neighborhood,  etc.  The  completion  of  these  tasks  provide 
the  individual  with  opportunities  to  activate  his  se"\f- 
assessment  schema.    Subsequently,  he  modifies  his  schema 
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to  account  for  his  task  performance  and  the  external 
factors  which  influenced  this  performance. 

In  addition,  internal  influences  (such  as  various 
individual  characteristics)  can  affect  the  components  of 
the  self -assessment  schema.  A  person's  temperament,  such 
as  whether  one  is  an  introvert  or  an  extrovert,  may 
affect  how  the  self-assessment  schema  is  utilized  in  a 
given  task.  In  addition,  an  individual's  self-esteem  or 
self -concept,  motivation,  school  achievement,  social 
skills,  etc.  also  play  a  role  in  influencing  the 
components  and  formation  of  this  self -assessment  schema, 
thus  ultimately  influencing  the  individual's  performance 
in  a  task. 

Because  the  e  internal  and  external  influences  act 
as  mediators  of  th^  self-assessment  schema,  they  provide 
opportunities  or  avenues  to  intervene  by  modifying  the 
schema,  which  ,  in  turn,  may  change  an  individual's 
performance  on  a  task.  Moreover,  the  interactions  among 
the  self-assessment  schema,  the  internal  influences  and 
the  external  influences  are  not  just  one-way 
interactions.  The  self-assessment  schema  are  not  only 
being  influenced  by  external  and  internal  influences. 
The  self-assessment  schema  (and  all  other  cognitive 
processes)  also  impact  on  these  external  and  internal 
factors.  For  instance,  modifying  an  individual's 
previously  ineffective  self-assessment  schema  to  a  more 
effective  schema  may  improve  his  self-concept  and  his 
self-confidence  (internal  characteristics),  as  well  as 
also  improve  his  performance  at  school  (external 
environment) . 

Conclusions  and  Implications  for  Research 

This  paper  provided  a  brief  discussion  of  the 
metacognitive  literature  and  the  social  psychology 
literature  as  it  may  pertain  to  self-assessment  as  a 
social  cognitive  construct.  These  theoretical  frameworks 
posited  (mainly  Flavell's  metacognitive  components  and 
the  social  cognitivists  (Vygotsky,  Cole  and  Scribner) 
provided  the  basis  for  a  proposed  model  oil  self- 
assessment  in  an  academic  environment.  Much  of  the  self- 
assessment  literature  out  there  seems  to  be  limited  to 
describing  the  manifestations  of  self-assessment 
behaviors  in  various  student  populations.  The  utility  of 
providing  a  model  of  self-assessment  is  that  it  provides 
a  contextual  framework  in  which  we  could  discuss  various 
self -assessment  behaviors  in  a  less  limited  manner.  How 
can  we  observe  developmental  change  in  self-assessment? 

Brown  and  DeLoache  (1978),  using  recommendations 
made  by  Cole  and  Scribner  (1975),  advocated  three 
components  of  researching  developmental  change  in 
competencies.     It  calls  for  a  synthesis  of  ethnographic 
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and  experimental  methods  and  for  the  investigation  of 
particular  activities  along  a  range  of  situations  (from 
natural  observations  to  experimental) .  First,  they 
believe  one  should  examine  the  individual's  understanding 
of  the  experiment  or  task  and  his  role  in  this.  One 
should  be  fully  aware  of  the  task  demands  and  how  they 
appear  to  the  individual  being  tested.  Second,  they 
believe  in  "experimenting  with  the  experiment".  That  is, 
researchers  should  use  variations  of  a  paradigm 
applicable  to  the  individuals  being  studied.  Third, 
researchers  should  investigate  the  same  process  in  a 
range  of  situations,  whether  in  natural ly-occurrring  or 
experimental  situations. 

I  believe  the  current  self-assessment  literature  can 
benefit  from  conducting  research  in  this  manner.  The 
complexity  of  self-assessment  and  its  importance  to 
learning  behooves  us  to  make  certain  that  research  in 
this  area  goes  beyond  merely  descriptive  analysis  of 
self -assessment  behaviors.  The  development  of  a  viable 
model  for  self-asessment  can  be  instrumental  in  enhancing 
our  understanding  of  what  exactly  develops  when  one  goes 
from  a  novice  learner  to  an  experienced  learner  in  a 
knowledge  domain. 
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The  Perception  of  the  Educational  Value  of  Channel  One  Among  Secondary 

Level  Teachers  and  Students. 

Ted  Carlin  Zulma  Quinones  Robert  Yonker 


Whittle  Communication's  Channel  One  is  an  educational  broadcast  service  destpned 
specifically  for  teenagers.  This  twelve  minute  daily  broadcast  claims  to  deliver  top-quality 
news  and  information  to  schools,  via  satellite,  with  state-of-the-art  production 
techniques  '.  By  creating  a  program  that  is  completely  original,  Channel  One  hopes  to 
consistently  deliver  riveting,  age-appropriate  coverage  to  an  audience  lacking  in  consistent 
rtcvv  viewership.  The  purpose  of  this  study  was  to  describe  student  and  teacher 
attentiveness  to  Channel  One,  and  the  perceived  educational  value  and  the  quality  of  the 
presentation  of  Channel  One  within  the  Lakota  School  District  of  northwestern  Ohio.  A 
self-report  questionnaire  was  used  to  survey  all  students  (n=512)  and  teachers  (n=33)  at  the 
junior  and  senior  high,  schools.  The  results  show  that  the  mean  viewing  time  of  Channel 
One  by  students  is  8.5  minutes,  and  that  74.8  %  of  the  students  supported  Channel  One  s 
claim  that  the  service  provides  new  ideas  to  think  about.  However,  78.9%  of  the  students 
do  not  seek  out  more  information  about  a  topic  shown  on  Channel  One.  In  terms  of  the 
presentation  of  Channel  One,  72.6%  of  the  students  rated  the  overall  quality  of  the 
programming  to  be  good  or  excellent.  The  average  viewing  time  for  teacners  was  10 
minutes,  with  87.9%  indicating  that  Channel  One  provides  new  ideas  to  thinK  about. 
Ninety-seven  percent  of  the  teachers  staled  that  the  overall  quality  of  Channel  One  was 
good  or  excellent.  Overall,  the  analysis  of  the  data  seems  to  support  the  effectiveness  of 
Channel  One  in  increasing  students'  awareness  of  news  and  current  events. 

Introduction 

Studies  and  commentaries  ("A  Nation  Still  at  Risk,"  1989:  Harris,  1989), 
suggested  that  American  education  is  in  serious  trouble  and  that  graduates  of  American  high 
schools  are  not  competitive  with  their  counterparts  in  many  foreign  countries  Educators, 
politicians,  and  social  critics  have  offered  reasons  for  the  problems  with  education,  but 
most  all  agreed  that  American  students  are  deficient  in  areas  such  as  science,  mathematics, 
ceo<™hv,  history,  and  cultural  literacy.  Particularly  troublesome  to  many  was  the  tact 
thaui  number  of  students  had  little,  if  any,  awareness  of  basic  current  events  and  issues. 
The  education  plight  is  reflected  in  some  of  the  following  educational  indicators: 

*Six  percent  of  seventeen-year-old  high  school  students  can  solve  multi-step 
math  problems  and  use  basic  algebra. 

*Seven  percent  are  able  to  infer  relationships  and  draw  conclusions  from  detailed 
scientific  knowledge. 

♦Given  a  blank  map  of  Europe  and  asked  to  identify  particular  countries,  young 
American  adults  typically  give  the  correct  answer  less  than  one  time  in  four  (Finn, 
1989) 

*Twcntv-seven  million  adults  in  the  United  States  are  functionally  illiterate. 
*Fortv-six  million  adults  are  considered  "marginal"  literates. 
The  drop-out  rate  ranges  between  40  and  60  percent  in  arers  like  Los  Angeles, 
Boston,  Chicago,  Detroit,  and  New  York. 

*Each  day,  nearly  3800  teenagers,  nationwide  drop  out  of  school,  according  to  the 
Department  of  Education  (Townley,  1989). 

From  the  educational  indicators  presented,  it  is  clear  that  the  panel  which  produced 
the  "Nation  at  Risk"  report  in  1983  had  good  reason  to  be  alarmed.  The  panel  indicated 
that  the  United  States  had  to  act  promptly  or  an  educational  meltdown  would  occur,  the 
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educational  svstein  would  crack,  our  culture  erode,  our  economy  totter,  our  national 
defenses  wcake,  ^  W  ^  emer  „  f      the  Second 

Gallun  Ph i  Delia  KaZa  Poll,  which  surveyed  the  attitudes  of  U.b.  teachers  toward  the 
nuWicPschool^  TeaK  tend  to  regard  themselves  as  martyrs.  They  believe  that  they  are 
Snaoor"and undeire  warded,  and  they  blame  everyone  but  themselves  for  recognized 
unappreuaicu  ai.u    u  j  ,   mentioned  reason  for  the  public 

SC£°°  dve l^to  ai  d5J^52^  teachers  (830  respondents-57.2% 

^^^^S^^^  4.2%  both,  5%  other)  in  the  1989  Gallup 
Mtffirf  interest  and  support.  Others  charged  that  parents  do  not  help 
studen  sPreaJize  £  importance  of  preparing  for  the  future.  Some  pointed  to  parenta 
aoathv  whUe  others  said  that  parents  lack  faith  in  the  school  system.  Lack  of  proper 
financial Support  was  the  second  most  frequently  mentioned  reason,  teachers  acknowledge 
fnafthTlac^ of  financial  support  is  related  to  other  school  problems.  Other  reasons 
mm  onid  as  factors  leadings  the  public  education  system's  Possible  meltdown  were. 
«»niic'  lark-  nf  interest/truancv  (26%),  poor  curriculum/poor  standards  (b,o), 

cXLiW 

(3%)  difficult  in  getting  good  teachers  (2%),  and  low  teacher  salaries  (7%). 
(  h  A  semi- literate  population,  lacking  both  good  learning  habits  and  basic  skills  and 
knowledg  cannot  support  a  productive  economy.  Scholars  and  studies  often  speak  of 
potential  so  utions  which  can  be  used  as  guides  to  improve  the  ^^^^^ 
Finn  (1987)  offered  direction  for  the  improvement  of  the  educational  system.  Some  of  his 
recommendations  indicate  that  we  must: 

*Focus  public  reflation  of  education  on  ends,  not  means-professional  educators 
need  the  latitude  to  organize  schools  and  determine  what  happens  within i  them. 
*Let  the  schools  manage  them  selves-each  school  should  make  its  own  key 
educational  management  decisions. 
*  Promote  more  imaginative  school  leadership. 

*Enea«e  Darents  as  well  as  schools.  , 
*Make%etter  use  of  technology-technology  allows  the  teacher  to  transform  the 
classroom  into  a  place  where  teachers  teach,  children  learn. 

Finn's  final  recommendation-make  better  use  of  technology-did  not  fall  on  deaf 
ears  The  classroom  has  been  profoundly  affected  by  technological  innovation,  from 
comm. ters  to  televisions.  The  use  of  these  new  innovations  create  new  environments  and 
ScwSS  of  Sedge.  The  classroom  is  vulnerable  to  technological  innovauon  (Kaha 
1990)  and  as  Finn  0987)  has  indicated  the  teacher  should  make  better  use  o  the 
technology  As  Kaha  (1990)  states:  "The  classroom  cannot  exclude  the  impact  of 
As  Marshall  Mcluhan  pointed  out,  one  can  be  absorbed  by  the  glowing  screen 
of  television  in  a  wav  that  is  not  possible  with  books. 

of  television  in  a  ^  ^  ^  ^  ^  conducted  ^  d    f  th 

effects  of  television  on  North  American  children,  as  reported  in  Television  in  jhg  Livesiit 
^^niSrn^n  The  primary  focus  of  this  study  was  on  the  uses  and  functions  of 
SS^n^fflca^e.  of  children.  The  study  found  that  public  affairs  and  news 
proems  were  not  viewed I  consistency  until  the  teenage  year.  Viewing  of  these  VP*  of 
nroerams  depended  on  the  teenager's  age,  sex,  and  mental  development  Their  results 
S£  that  the  "brighter"  teens  were  the  first  to  turn  away  from  ^*^™^n™* 
nroerams  in  favor  of  news  and  public  affairs  content.  These  results  did  not  ment.on  the 
use^  televisionTn  the  classroom,  but  it  did  recommend  tailoring  television  programming 
to  the  desired  audience  to  achieve  maximum  cognition  and  development.  ,  ,  .  .  tn 
to  tne  aesirea  au  ^  ^  hQw  [q  mQ  ^  instrucnonal  television  to 

teach  teens  and  in  turn  thev  v. ere  still  learning  how  to  integrate  video  into  their  curricula 
(Landay  1989)  Dunn*  this  period  Walter  Cronkite  was  inspired  by  the  vision  of  a  unique 
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offering  to  the  nation's  young,  specifically  through  his  PBS  series.  Cn  nkite  dreamed  of 
linkme  die  power  of  television  to  the  American  public  education  system  He  wanted  to 
instalf satellite  antennas  on  the  roof  tops  of  the  nation's  high  schools  to  dehver  to  as  many 
students  as  possible  the  nation's  best  teachers  (Landay,  1989).  From  Cronkite  s  idea, 
sSte  Educational  Services,  Inc.  was  created  in  the  1970s  and  began  pr rtucinp Why  In 
The  World  in  the  1980s.  Cronkite  was  not  the  only  person  who  had  great -visions  o\ 
television's  power:  The  use  of  television  in  the  classroom  had  emerged  and  the  bat  le 
Stween  the  big  communication  companies  had  begun.  Ted  Turner,  the  cable-ready  King 
with  CNN  Newsroom,  and  Christopher  Whittle,  an  innovative  leader  of  the  pnnt  industry 
with  CHANNEL  ONE,  are  competing  for  supremacy. 

While  Turner,  arguably,  needs  no  introduction,  Whittle  may.  CHANNEL  ONE  s 
distributed  by  Whittle  Communications  L.P.  of  Knoxville,  Tennessee.  Christopher  Whittle 
heads  a  media  empire  which  is  valued  at  more  than  $400  million.  In  March  of  19A), 
Whittle  began  offering  a  daily  twelve-minute  program  of  current  events  and  news. 
CHANNEL  ONE  has  five  educational  goals: 

*To  enhance  cultural  literacy. 

*To  promote  critical  thinking. 

*To  provide  a  common  language  and  shared  experience. 

*To  provide  relevance  and  motivation. 

*To  strengthen  character  and  build  a  sense  of  responsibility. 

CHANNEL  ONE  claims  to  offers  "a  unique  combination  of  elements,"  such  as: 

*  Instructional  value-CHANNEL  ONE's  format  incorporates  a  variety  of  factual 
information  that  is  related  with  a  teenage  slant  to  current  events. 
♦Video  medium-featuring  the  use  of  graphics  and  imaging  techniques  familiar  to 
today's  teens. 

*The  immediacy  aspect  that  is  generated  by  a  broadcast  news  format.  The  news  is 
presented  in  a  language  geared  to  teens.1 

These  elements,  along  with  others,  are  offered  to  junior  high  and  high  schools. 
The  schools  receive  $50,000  worth  of  hardware  (a  satellite  dish,  a  VCR-based  recording 
and  plavback  station,  and  televisions)  in  exchange  for  the  guarantee  that  the  newscast  will 
be  aired  on  a  dailv  basis  and  viewed  by  the  school.  Of  the  twelw  minutes  beamed  to 
students  in  classrooms,  two  minutes  are  commercials.  As  of  March  1991,  which  marked 
CHANNEL  ONEs'  first  anniversary,  8270  (Sylvester,  1991)  schools  were  signed  up  for 
the  standard  three-vear  contract.  After  the  three  year  period,  the  school  can  renew  its 
contract  or  decide  to  terminate  their  relationship  with  Channel  One,  at  which  time  the 

equipment  will  be  returned. 

Whittle  Communications  re;  iizes  that  children  generally  do  not  watch  news,  l  ney 
are  using  technology  in  an  effort  to  link  news  to  the  educational  system,  and  thereby 
improve  the  educational  system.  Therefore,  the  purpose  of  this  study  was  to  describe 
teachers'  and  students'  attentiveness  to  Channel  One,  the  perceived  educational  value,  and 
the  quality  of  presentation  of  CHANNEL  ONE. 

Methods 

The  school  district  from  which  the  subjects  were  obtained  is  the  largest  school 
district  in  land  area,  in  the  state  of  Ohio.  Channel  One  has  been  broadcasnng  in  the  senior 
high  school  since  September,  1990,  and  in  the  junior  high  school  since  January,  1991. 


]Notc:     Channel  One  Infoi  nation  obtained  from  the  Chai  ncl  One  Media  Kit, 
prepared  by  Whittle  Communication,  Inc..     Knoxville,  Tennesse. 
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E  collar  industries  and  agriculture.  The  per  cap.B  .ncorne  .n  988,  wa 

ITo"  Estate  average  was  $29,195).  The  1990  Vital. Statics  on  Ohio  School  D.stncts 
ndicaS  S 96.9%  of  the  residents  of  this  school  dismctw ere  Can  as.  n 

The  school  district  contains  one  senior  high  school  (grates  v  i-j  j 
high  school  grades  7  &  8),  from  which  the  "W^^S 

responded  to  the  questionnaire  by  gender,  within  each  grade. 


Insert  Table  1  about  here 


There  were  512  responses  from  600  questionnaires  distributed  (85.3%)  to  students 

distributed  to  teachers  in  both  schools,  with  15  responses  from  male  teacher.,  All  subjects 
were  eiven  the  option  of  not  participating  in  the  study.  marchers  one  for 

Two  self-administered  questionnaires  developed  by  the  researchers  one :  ioi 


reSearCEarch  of  the  two  questionnaires  was  one  page  in  length,  and  was  divided I  into  four 

One's  statements  of  their  intended  educational  goals  for  the  program.*  In  the  third  section 

ThS^fffl  SI  the  subjects'  perception  of  various  ae.thetic :  qua  hues  of 
Channel  One,  as  well  as  their  perceptions  of  the  information  content  and  its  delivery  ( see 
Zettl,  1990,  for  a  complete  description  of  these  characteristics). 

Findings 

The  averaee  viewing  time  of  Channel  One  by  the  students  in  this  study _  was £5 
minutes  of  the  12  minutes  available  (71.8%).  The  average  viewing  *™°[™™^f 
b)  teach  rs  was  10  minutes  (83.3%).  These  findings  indicate  a  relatively  high  degree  of 


W  Channel  One  Information  obtained  from  the  Channel  One  Media  Kit, 
prepared  by  Whittle  Communication.  Inc..     Knoxvillc.  rennesse. 
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viewing  by  both  groups  of  subjects.  The  high  degree  of  attentiveness  may  be  attributed  to 
the  recent  implementation  of  Channel  One  in  the  senior  high  school  (September  1990)  and 
the  junior  high  school  (January  1991).  Thus,  it  is  possible  that  the  newness  of  the  service 
may  be  creating  the  high  interest  level.  Given  that  at  the  present  time,  there  is  no  other 
available  evidence  with  which  to  compare  these  result  a  longitudinal  study  of  Channel  One 
viewing  attentiveness  would  provide  a  necessary  benchmark  for  future  comparisons. 

The  responses  related  to  the  attentiveness  to  the  various  components  of  the 
broadcast  as  well  as  the  perceived  educational  value  of  Channel  One  are  shown  in  Table  2. 
The  responses  were  obtained  via  yes  or  no  answers  and  are  reported  interms  of  the  percent 
of  "yes"  responses. 


Insert  Table  2  About  Here 


The  responses  obtained  in  Table  2  should  be  considered  together  because  the 
subjects'  degree  of  attentiveness  contributes  to  the  perceived  educational  value  of  Channel 
One  A  high  degree  of  attentiveness  (over  65%)  was  reported,  with  the  exception  of  the 
viewing  of  the  commercials  (51.2%).  This  may  be  attributed  to  the  repetition  and  lack  of 
originality  in  the  commercials.  Further  study  of  the  commercials  within  Channel  One  may 
provide  more  concrete  information.  Under  the  variable  of  perceived  educational  value,  the 
information  on  Channel  One  is  not  generally  used  in  classroom  discussion  by  the  students 
(38  1%),  nor  in  their  out-of-school  activities  (33.7%).  The  students  very  rarely  seek  out 
additional  information  about  topics  discussed  on  Channel  One  (20.4%),  and  they  rarely  ask 
questions  about  this  information  (30.8%).  On  the  other  hand,  Channel  One  does  provide 
new  ideas  to  think  about  (72.7%),  and  it  does  make  students  more  aware  of  other  cultures 
(77  3%)  Sixty  percent  indicated  that  Channel  One  does  provide  positive  role  models. 
Therefore,  for  this  group  of  students,  the  data  supports  Channel  One's  original 
programming  goals  of  enhancing  cultural  literacy  and  promoting  critical  thinking.  Students 
in  all  grade  levels  are  receiving  new  information  from  Channel  One,  especially  dealing  with 
other  cultures.  This  cultural  information  is  most  often  shown  during  the  Feature  segment, 
which  is  also  the  highest  viewed  segment  of  the  program.  However,  it  does  not  support 
the  goal  of  providing  relevance  and  motivation,  as  students  do  not  pursue  additional 
information  outside  of  the  Channel  One  environment.  This  could  reflect  a  continuation  of 
their  personal  viewing  habits  of  television  at  home. 

With  respect  to  the  quality  of  the  presentation  of  Channel  One,  the  responses  are 
tabulated  in  Table  3.  This  table  summarizes  both  the  junior  and  senior  high  school 
students'  evaluation  of  the  program's  visual,  aural,  and  message  content. 


Insert  Table  3  About  Here 


Table  3  summarizes  the  students'  perceptions  of  the  quality  of  the  presentation  of 
Channel  One.  Approximately  72%  of  the  students  rated  the  overall  quality  of  Channel 
One's  presentation  to  be  good  or  excellent.  Channel  One  was  also  given  high  marks  in  its 
ability  to  communicate  information  to  the  viewer  (77%  good  or  excellent).  In  the  areas  of 
visual  style  (graphics,  maps  and  charts,  colors,  on-screen  lettering),  Channel  One  received 
high  marks  again'(no  less  than  75%  good  or  excellent).  The  use  of  student  and  celebrity 
reporters  also  fared  well  with  the  students  (67%  good  or  excellent).  However,  the  pacing, 
music  and  visual  excitement  of  the  program  was  not  well  received  by  the  students  (no  more 
than  56%  good  or  excellent).  Therefore,  Channel  One  appears  to  provide  common 
knowledge  and  shared  experiences  through  familiar  production  techniques  and 
personalities.  Many  students  commented  that  the  style  of  Channel  One  is  similar  to  that  of 
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programs  on  MTV  and  Nickelodeon,  which  use  large  on-screen  lettering,  visually  exciu 
graphics,  and  identifiable  and  likable  person  lities.  Some  of  the  lack  of  visual  excitemont 
may  be  attributed  to  the  news  content  of  the  program. 

In  Table  4,  the  teachers'  degree  of  attentiveness  and  perceived  educational  value  of 
Channel  One  are  tabulated.  Table  5  summarizes  the  teachers'  evaluation  of  the  program's 
visual,  aural,  and  message  content. 

Insert  Table  4  About  Here 


Insert  Table  5  About  Here 


The  responses  in  Tables  4  and  5  summarize  the  teachers'  degree  of  attentiveness 
and  the  preceived  educational  value  and  quality  of  the  presentation  of  Channel  One.  All  of 
the  Channel  One  program  segments  were  highly  viewed  by  the  teachers  (no  less  than  87% 
attentiveness),  except  for  the  commercials  (42%).  This  may  indicate  that  the  teachers  find 
the  commercials  to  be  of  little  value  to  them.  This  is  not  really  surprising,  since  the 
commercials  are  targeted  toward  the  younger  student  audience.  As  for  perceived 
educational  value,  the  teachers  also  highly  rated  Channel  One's  performance  in  cultural 
awareness  (100%),  providing  positive  role  models  for  students  (97%),  and  providing 
students  new  ideas  to  think  about  (88%).  The  teachers,  like  the  students,  reported  that 
Channel  One  was  not  generally  being  used  in  classroom  discussion  (58%)  or  to  motivate 
them  to  seek  out  additional  information  on  Channel  One  topics  (42%). 

Overall,  97%  of  the  teachers  gave  Channel  One's  presentation  a  grade  of  good  or 
excellent.  The  highest  rated  elements  were  again  the  visual  style  elements  of  graphics, 
lettering,  etc.  (no  less  than  94%  good  or  excellent).  Again,  the  commercials  were  not  very 
well  rated,  with  only  54%  giving  them  a  grade  of  good  or  excellent. 


Summary 

Channel  One  appears  to  be  accomplishing  most  of  its  intended  programming  goals 
with  its  unique  format.  Students  and  teachers  responded  positively  to  Channel  One's 
ability  to  provide  new  ideas  in  the  learning  environment,  to  enhance  cultural  literacy,  and  to 
provide  positive  role  models.  Channel  One  also  has  been  able  to  effectively  reach  the 
teenager  with  programming  that  they  like  and  are  familiar  with.  It  has  been  able  to  get 
students  interested  and  involved  in  ideas  and  issues  in  which  they  may  have  not  been 
previously  exposed.  However,  Channel  One  falls  short  in  its  ability  to  promote  motivation 
outside  the  Channel  One  environment .  This  may  be  caused  by  the  lack  of  follow-up  on 
issues  and  concepts  by  teachers  in  the  classroom.  Instructional  material  is  provided  with 
the  program,  but  it  appears  that  it  is  not  being  utilized  in  the  classroom.  As  Channel  One 
becomes  an  established  part  of  the  school  day  in  the  future,  it  may  become  more  useful  to 
teachers  and  students.  .  , 

Future  research  on  Channel  One  may  be  able  to  address  these  concerns.  The  timing 
of  this  study,  shortly  after  the  implementation  of  Channel  One  in  these  schools,  may  have 
obtained  more  positive  results  than  expected  because  of  Channel  One's  newness  in  the 
learning  environment.  Future  studies,  especially  longitudinal  in  design,  would  enable 
researchers  to  examine  this  concern.  Future  studies-might  also  examine  a  number  of 
different  school  settings  where  Channel  One  is  present  to  possibly  construct  valid 
generalizations  about  the  use  of  Channel  One  as  an  educational  tool.  Another  area  for 
future  research  is  the  effectiveness  of  the  commercials  within  the  program.  Although  this 
study  found  that  the  commercials  were  not  generally  viewed,  their  impact  on  this  young 
audience  could  be  analyzed  in  greater  detail. 
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TABLE  1 
DESCRIPTION  OF  SUBJECTS 


TOTAL 
MALES 
FEMALES 
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TOPIC 


TABLE  2 

THE  STUDENTS'  DEGREE  OF  ATTENTIVENESS  AND 
THE  PERCEIVED  EDUCATIONAL  VALUE  OF  CHANNEL  ONE 
(PERCENTAGE  OF  YES  RESPONSES,  n=512) 

QUESTION 


ATTENTIVENESS      Do  you  typically  watch  the  Lead  story? 

Do  you  typically  watch  the  Focus  segment? 
Do  you  typically  watch  the  Feature  segment? 
Do  you  typically  watch  the  Pop  Quiz? 
Do  you  typically  watch  the  End  piece? 
Do  you  typically  watch  the  commercials? 


PERCEIVED 
EDUCATIONAL 
VALUE 


Is  the  information  on  Channel  One  generally  used 
in  classroom  discussion? 

Is  the  information  on  Channel  One  generally 
useful  in  your  out  of  school  activities? 
Does  Channel  One  provide  new  ideas  to  think 
about? 

Do  you  ever  ask  questions  about  the  information 
shown? 

Do  you  ever  seek  out  more  information  about  a 
topic  shown? 

At  time,  docs  Channel  One  relate  to  what  you  are 
currently  studying  in  class? 
Does  the  information  on  Channel  One  make  you 
more  aware  of  other  cultures? 
Does  Channel  One  provide  you  with  positive  role 
models?   


JR. 

SR. 

HIGH 

HIGH 

TOTAL 

n=179 

n=333 

n=512 

77.6 

78.9 

78.4 

62.3 

66.5 

65.0 

79.8 

78.6 

79.0 

80.9 

72.1 

75.2 

61.2 

67.6 

65.4 

61.7 

45.4 

51.2 

42.1  35.9 


38.1 


41.5 

29.3 

33.7 

75.4 

71.2 

72.7 

41.5 

24.9 

30.8 

28.4 

16.0 

20.4 

68.3 

76.8 

73.9 

78.1 

76.8 

77.3 

63.9 

58.4 

60.4 
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TABLE  3 

THE  QUALITY  OF  THE  PRESENTATION  OF  CHANNEL  ONE 
(PERCENTAGE  OF  GRADE  GIVEN  TO  EACH  ITEM) 

tot.,  n  -  m  ™  H.ofc.ff,  '^Sn/Ll  m  skspe 


ITEM 


The  ability  of  Channel  One  to  effectively 
communicate  the  information  to  the  viewer 

The  style  of  graphics  used 


Colors  used  during  Channel  One 


Maps  and  charts  used 


On-screen   lettering  used 


Type  of  music  used 


Loudness  of  music  used 


The  pacing  of  Channel  One 


Visual  excitement 


The  commercials 


The  hosts  of  Channel  One 


The  adult  reporters 


The  student  reporters 


The   celebrity  reporters 


The  overall  quality  of  the  presentation 


Jr.  High 
Sr.  High 

Total 
Jr.  High 
Sr.  High 

Total 
Jr.  High 
Sr.  High 

Total 
Jr.  High 
Sr.  High 

Total 
Jr.  High 
Sr.  High 

Total 
Jr.  High 
Sr.  High 

Total 
Jr.  High 
Sr.  High 

Total 
Jr.  High 
Sr.  High 

Total 
Jr.  High 
Sr.  High 

Total 
Jr.  High 
Sr.  High 

Total 
Jr.  High 
Sr.  High 

Total 
Jr.  High 
Sr.  High 

Total 
Jr.  High 
Sr.  High 

Total 
Jr.  High 
Sr.  High 

Total 
Jr.  High 
Sr.  High 
Total 


23.4 
33.4 
29.8 


43.0 
38.4 
39.6 


52.5 
42.5 
45.6 


44.9 
51.1 
48.1 


42.8 
52.7 
48.1 


22.9 
19.6 
20.4 


27.4 
20.6 
22.5 


20.8 
18.2 
18.8 


23.2 
19.3 
20.4 


30.3 
23.4 
25.2 


B 


D 


47.4 
47.6 
47.5 


25.1 
15.4 
18.7 


2.3 
2.7 
2.5 


34.6 
40.2 
38.1 


18.9 
18.1 
18.7 


1.7 
1.2 

1.3 


27.1 
36.8 

23.1 


15.8 
16.1 
15.8 


2.2 
2.4 
2.3 


34.1 
30.5 
31.7 


15.3 
13.3 
14.0 


3.9 
4.2 
4.0 


30.8 
25.8 
27.1 


16.6 
13.2 
14.2 


4.0 
4.3 
4.0 


26.8 
29.1 
27.9 


33.5 
27.5 
29.4 


7.8 

10.4 

9.4 


33.7 
35.5 
34.2 


28.6 
30.6 
29.4 


6.3 
5.8 
6.2 


31.4 
32.2 
31.5 


35.9 
38.6 
37.5 


6.2 
6.9 
6.7 


37.3 
30.7 
32.9 


25.4 
31.3 
29.0 


6.2 

10.8 

9.4 


25.7 
25.6 
25.6 


28.0 
25.9 
26.8 


7.4 

11.7 

10.0 


F 


1.7 
0.9 
1.2 


1.7 
2.1 
2.1 


2.2 
2.1 
2.1 


1.7 
0.9 
1.2 


5.7 
3.9 
4.4 


8.9 

13.4 

11.7 


4.0 
7.6 
6.3 


5.6 
3.9 
4.4 


7.9 
7.8 
7.7 


8.6 

13.3 

11.2 


25.4 

42.4 

22.0 

3.4 

6.8 

25.8 

38.7 

22.5 

5.1 

7.8 

25.6 

40.0 

22.3 

4.4 

7.3 

20.8 

47.9 

21.4 

4.0 

5.8 

23.2 

41.2 

27.1 

4.9 

3.6 

21 .7 

42.7 

24.6 

4.6 

4.2 

31.0 

42.5 

19.5 

1.7 

5.2 

29.6 

35.7 

23.2 

7.6 

3.9 

29.8 

37.3 

21.5 

5.4 

4.2 

31.2 

40.5 

20.6 

2.9 

4.6 

27.5 

38.5 

21.4 

7.6 

4.8 

28.1 

38.8 

20.8 

6.0 

4.7 

29.9 

44.1 

19.2 

2.3 

4.5 

26.2 

45.2 

21.1 

3.9 

3.6 

27.5 

44.6 

20.2 

3.3 

3.8 

1  0 
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TABLE  4 

THE  TEACHERS'  DEGREE  OF  ATTENTIVENESS  AND 
THE  PERCEIVED  EDUCATIONAL  VALUE  OF  CHANNEL  ONE 
 (PERCENTAGE  OF  YES  RESPONSES,  n  =  33)  


TOPIC 


QUESTION 


PERCENT 
100.0 


ATTENTIVENESS 


PERCEIVED 
EDUCATIONAL  VALUE 


Do  you  typically 
Do  you  typically 
Do  you  typically 
Do  you  typically 
Do  you  typically 
Do  you  typically 


watch  the  Lead  Story  segment? 
watch  the  Focus  segment? 
watch  the  Feature  segment? 
watch  the  Pop  Quiz  segment? 
watch  the  End  Piece  segment? 
watch  the  commercials? 


Is  the  information  on  Channel  One  generally  used 
in  classroom  discussion? 

Is  the  information  on  Channel  One  generally  useful 

in  your  out  of  school  activities? 

Docs  Channel  One  provide  new  ideas  to  think 

about? 

Do  you  ever  ask  questions  about  the  information 
shown  on  Channel  One? 

Do  you  ever  seek  out  more  information  about  a 

topic  shown  on  Channel  One? 

At  times,  does  Channel  One  relate  to  what  you  are 

currently  studying  in  class? 

Does  the  information  on  Channel  One  make  the 

students  more  aware  of  other  cultures? 

Docs  Channel  One  provide  the  students  with 

positive  role  models?  


87.9 
100.0 
93.9 
87.9 
42.4 


57.6 

63.6 

87.9 

78.8 

42.4 

78.8 

100.0 

97.0 
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TABLE  5 

QUALITY  OF  THE  PRESENTATION  OF  CHANNEL  ONE 
(PERCENTAGE  OF  GRADE  GIVEN  TO  EACH  ITEM) 

n  =  33  


ITEM 


The  ability  of  Channel  One  to  effectively 

communicate  the  information  to  viewers 

The  style  of  graphics  used 

Colors  used  during  Channel  One 

Maps  and  charts 

On-screen  lettering 

Type  of  music  used 

Loudness  of  music  used 

Pacing  of  Channel  One 

Visual  excitement 

The  commercials 

The  hosts  of  Channel  One 

The  adult  reporters 

The  student  importers 

The  celebrity  reporters 

The  overall  quality  of  the  presentation  


PERCENTAGE  OF  GRADE 


A 

B 

c 

60.6 

36.4 

3.0 

57.6 

42.4 

66.7 

27.3 

6.1 

69.7 

27.3 

69.7 

27.3 

3.0 

51.5 

30.3 

12.1 

36.4 

45.5 

18.2 

33.3 

51.5 

15.2 

45.5 

45.5 

3.0 

9.1 

45.5 

33.3 

42.4 

39.4 

15.2 

42.4 

45.5 

12.1 

42.4 

45.5 

12.1 

42.4 

48.5 

9.1 

63.6 

33.3 

3.0 

D 


F 


3.0 
9.1 
3.0 


3.0 
3.0 
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An  Advance  toward  Instructional  Management: 
Prescriptive  Knowledge  Base  of  Learner  Control 


Jaesam  Chung 
Indiana  University 


Research  Background 

Learner  control  is  widely  believed  to  be  a  highly  desirable  feature  of 
interactive  learning  systems  because  learner  control  allows  learning  to  be 
individualized  to  each  learner's  needs.  Learner  control  of  instruction  is 
intuitively  appealing  since  it  is  assumed  that  learners  will  be  more  motivated 
if  allowed  to  control  their  own  learning. 

However,   research  findings  regarding  the  effects  of  learner  control 
have  been  inconclusive,  but  more  frequently  negative  than  positive  (Carrier 
&  Sales,  1987;  Steinberg,  1977;  Tennyson,  1980;  Ross  &  Morrison,  1989). 
Whereas  learner  control  of  instructional  options  has  a  great  deal  of  intuitive 
appeal,  and  an  enormous  amount  of  computer-based  program  design  has  been 
carried  out,  much  of  research  contradicts  unrestricted  learner  control 
(Hannafin  &  Peck,  1984). 

These  negative  findings  may  occur  because  many  students,  especially 
low  achievers,  lack  the  knowledge  and  motivation  to  make  appropriate 
decisions  regarding  such  conditions  as  pacing,  sequencing  of  content,  use  of 
learning  aids,  and  amount  of  practice.  These  negative  findings  may  also  exist 
because  most  learner  control  strategies  employed  in  research  have  been 
insufficient,  piece-meal,  or  narrow. 

Teachers  and  instructional  designers  may  require  more  systematic 
guidelines  for  implementing  effective  learner  control  in  real  world  situations. 
A  comprehensive,  integrated,  diverse,  prescriptive,  and  specific  theory  /model 
of  learner  control  that  is  viable  for  the  interactive  learning  systems  may  lead 
to  more  effective  implementation  of  learner  control. 


Research  Questions 

•  What  types  of  learner  control  are  available  in  learning  situations? 

•  What  are  the  condition  variables  and  the  outcome  variables  when 

learner  control  types  are  considered  as  the  method  variable? 

•  Mow  can  educators  and  instructional  designers  identify  the  functional 

relationship  between  learner  control  and  learner's  cognitive 
development,  between  learner  control  and  learning  domains,  and 
between  learner  control  and  learner's  motivation? 

•  What  can  be  done  to  ensure  that  learners  will  be  able  to  effectively 

control  and  regulate  their  own  learning  in  different  instructional 
systems  including  non-individualized  systems,  individualized  systems, 
intelligent  learning  systems,  and  hypermedia  systems? 
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Research  Procedure 

This  study  reviews  a  variety  of  literature  (both  empirical  and 
theoretical)  relevant  to  "iearncv  control."    Learner  control  strategies 
reviewed  by  the  researcher  ;ue  analyzed  and  classified  by  the  Conditions- 
Methods-Outcomes  [C-M-Oi  v^adigm  (Reigeluth,  1983). 

in  tMs  research,  a  variety  of  learner  control  methods  (including 
content  control,  sequence  control,  pace  control,  display  control,  internal 
processing  control,  conscious  cognition,  metacognition,  intelligent  advisor, 
uivisem^  !l  strategy,  r.lc.)  are  identified  and  integrated.  This  study  inductively 
integrated  the  current  knowledge  base  concerning  learner  control  into  a 
prescriptive  form  that  vwuW  i    useful  to  instructional  designers  and  teachers. 
This  '  tudy  present  u  v\<j.  'smnr  ,u$bord"of  individual  learner  control 
strategics  to  an*j  w*th  mdivid;.j)  learning  and  instructional  situations,  as  does 
Keller's  ARCS  model  of  motivational  design  (Keller  &  Kopp5  1987). 


Results 

The  current  instructional  prescriptions  of  learner  control  are  composed 
of:  (1)  thieo  instructional  outcomes;  (2)  three  categories  of  instructional 
condition  variables;  and  (3)  five  or  more  types  of  method  variables. 


insert  Figure  1  (Variables  of  Learner  Control)  about  here 


Instructional  Outcomes 

Instruction  is  judged  by  its  effectiveness,  efficiency,  and  appeal  in  the 
context  of  instructional  management  (that  is,  learner  control).  Reigeluth 
(1979,  1983)  classifies  learner  outcomes  into  three  categories:  effectiveness, 
efficiency,  and  appeal  of  the  instruction.  Four  important  aspects  of  the 
effectiveness  of  instruction  are  the  accuracy  ("error  rate"),  the  speed 
("performance  efficiency"),  the  degree  of  transfer,  and  the  length  of 
retention.  The  efficiency  of  instruction  is  usually  measured  by  the 
effectiveness  divided  by  the  student  time  and  the  cost  of  instruction.  The 
appeal  of  instruction  is  measured  by  learner's  appreciation  and  desire  for 
more. 

Inst  rue;  tional  Conditions 

The  selection  of  specific  learner  control  strategies  is  determined  by 
three  sets  of  condition  variables:  (1)  learner  characteristics;  (2)  learning 
objectives  (or  domains);  and  (3)  learning/instructional  system?;.  Learning 
domains  are  usually  considered  constraint  and  constraint  can  be  a  condition 
variable.  Learning  systems  are  frequently  regard*  *  as  a  method  variable,  but 
in  this  vStudy,  they  are  only  part  of  learning  environment  (i.e.,  condition 
variable) 

The  learner  characteristics  involve  the  learner's  cognitive 
developmental  stage  (readiness),  motivation  task  persistence,  aptitude,  ability, 
prior  knowledge,  metacognition.  age,  etc. 
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Learning  objectives  (or  domains  of  learning)  are  also  a  crucial 
condition  variable  (constraint)  of  learner  control.  Gagne"  (1985)  classifies 
human  learning  domains  into  five  categories:  intellectual  skills,  verbal 
information,  cognitive  strategies,  motor  skills,  and  attitudes.  Bloom's  (1961) 
three  domains  (cognitive,  affective,  and  psychomotor)  and  Merrill's  (1983) 
performance-content  matrix  (remember,  use,  and  find;  fact,  concept, 
procedure,  and  principle)  are  good  classifications  of  learning  domains. 
Reigeluth  (1989)  suggests  several  types  of  learning  including  memorization, 
understanding,  skills  application,  generic  skills,  and  affective  learning. 

Instructional/learning  systems  also  seem  to  be  an  integral  part  of 
learner  control.  We  need  to  tailor  learner  control  methods  to  individual 
instructional/learning  systems.    Instructional/learning  systems  include  non- 
individualized  system,  individualized  instructional  systems  (such  as  traditional 
CAI,  TICCIT),  intelligent  learning  systems  (such  as  ICAI),  and  Hypermedia 
systems. 

Instructional  Methods 

Method  variables  of  learner  control  in  current  theory  include  Merrill's 
(1984)  notion  of  content  control,  sequence  control,  pace  control,  display 
(strategy)  control,  internal  processing  control  such  as  conscious  cognition 
and  metacognition,  and  intelligent  advisor  (Merrill,  Li,  &  Jones,  1990).  In 
addition,  Tennyson's  (1985)  advisement  strategy  and  Reigeluth's  (1979,  1983) 
notion  of  macru-level  control  (such  as  sequencing,  selecting,  summarizing, 
synthesizing),  user  control,  and  system  control  can  also  be  considered  as 
learner  control  strategies. 

Prescriptions  of  Learner  Control 

Considering  condition,  outcome,  and  method  variables  of  learner 
control  altogether,  instructional  prescriptions  of  learner  control  were 
contrived  in  a  smorgasbord  fashion.  Below  is  a  list  of  learner  control 
prescriptions  which,  based  on  the  conditions,  will  optimize  the  desired 
instructional  outcomes. 

A.  (CONTENT  CONTROL  over  segments,  lessons,  units,  and  courses)  Provide 

learners  with  more  opportunities  to  make  content  selection  as  learners 
grow  older. 

e.g.)    -  junior  high  school  level  -  a  choice  among  2-3 
alternatives  in  addition  to  required  courses 
-  university  level  ~  almost  unlimited  options 

Al      If  one  is  good  at  content,  one  can  use  options  effectively 

Selection  of  topic  is  dependent  on  the  learner's  competencies  in 
the  topic.  (Judd,  Bunderson,  &  Bessent,  1970;  Sasscer  &  Moore, 
1984). 

A2      Learner  control  is  likely  to  work  well  compared  to  program 

control  when  higher  order  skills  rather  than  factual  information 
are  being  taught,  and  when  the  content  is  familiar  to  learners 
(Hannafin  &  Peck,  1988).  This  view  supports  Reigeluth's  (1989) 
point  that,  to  some  extent,  user  (learner)  or  system  (program) 
control  will  be  determined  by  instructional  objectives. 
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B.  (SEQUENCF  CONTROL)  Offer  opportunity  for  student  to  sequence  the 

objectives  within  a  particular  course  of  instruction  in  any  order  IF 
the  student  so  desires. 

Bl       DON'T  use  sequence  control  WHEN  the  objectives  or  contents 

(segments,  lessons,  and  units  of  instruction)  are  sequenced,  as  in 
linear  delivery  systems  (lecture,  videotape,  slide  presentations,  or 
movies)  which  make  the  reordering  of  content  components 
difficult. 

B2      Use  sequence  control  in  self-paced  individualized  materials 

(such  as  workbooks,  textbooks,  etc)  which  enable  learner  to  do. 

B3      Learners  use  prior  knowledge  to  determine  their  instructional 
sequence  (Mager,  1961;  Mager  &  Clark,  1963). 

C.  (PACE  CONTROL)  Offer  the  opportunity  to  study  a  given  unit  or  lesson  as  long 

as  needed  IF  learners  are  using  individualized  instruction  or  self-paced 
instruction  (Merrill,  1984). 

CI       Keep  moderate  level  of  control  over  learning  time  to  make 

improve  performance  of  learner  (Tennyson,  Park,  &  Christensen, 
1985). 

(2       If  learners  get  coached  practice  on  self-directed  study,  then  they 
can  increase  their  performance  and  motivation,  and  can  reduce 
instruction  time  (Campbell  &  Terry,  1963). 

D.  (DISPLAY  or  STRATEGY  CONTROL)  Present  a  single  subiect-matter  idea  . 

a  general  statement  (generality)  or 
a  specific  example 

Dl      WHEN  students  are  provided  a  mechanism  for  selecting  and 
sequencing   those  displays  which  they  feel  are  required  to 
understand   a  given  objective,  at  least  three  types  of  display 
control  are  involved. 

-  selection  of  the  number  of  displays  required  for  perceived 
mastery 

-  control  of  sequence  of  different  types  of  displays 

-  selection  of  various  displays. 

D2      Use  display  selection  and  sequence  control  in  TICCIT  system  by 
means  of  rule,  example,  and  practice  button. 

E  (INTERNAL  PROCESSING  CONTROL)  Use  a  variety  of  internal  processing 
.strategies  WHEN  the  learner  interacts  with  the  presentation  of  the 
instruction. 

Interaction  of  these  internal  processes  with  the  stimulus  materials  will 
help  the  learner  perform  effectively  and  efficiently. 
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M       (CONSCIOUS  COGNITION)  In  order  to  encode  the  information 
presented  by  a  given  display,  learner  should  use  mental 
processes. 

In  order  to  compensate  for  inadequate  display,  learner  should 
appropriately  select  the  cognitive  processing  activities. 

rehearsal,  repetition,  paraphrasing, 
imaging,  encoding  via  mnemonic, 
exemplifying,  and  covert  prac  tice. 

Use  the  introspective  interview  with  learners  in  order  to  obtain  a 
record  of  the  processing  attempted  for  each  display. 

E2       (METACOGNITION)  In  order  to  guide  their  interaction  with  an 

instructional  system  being  use  ',  learners  should  use  the  "how  to 
study  or  learn"  model. 

When  learners  are  taught  to  use  more  appropriate  methods, 
they  acquire  not  only  subject  matter  but  also  some  internal 
theory  about  "how  f>  learn". 

Use  the  introspective  interview  technique  in  order  to  assess  the 
mata-cognitive  models  being  used  by  a  student. 

F.  (ADVISOR  STRATEGIES  in  new  CD'H  Ihe  an  Intelligent  Advisor  in  order  to 

determine  if  a  given  learner  needs  a  particular  type  of  interaction 
("transaction"),  which  wc*  not    originally  included  in  the  transaction 
frame  sets  of  the  course. 

G.  Individual  Difference 

Gl       If  hij?h-ab:Uty  students  are  put  under  learner  control  group  and 
lov\  a!  ; ;;•.>•  students  are  put  under  external  control  (Program 
Control;,  thr  performance  of  both  sides  will  be  improved. 

(Gwuie,  1 976) 

G2      Stud,  at  learning  should  be  individualized  because  students 
difi'er  in  the  way  they  use  instructional  options  in  learner 
control  conditions  (Mager  &  McCann,  1961:  Seidel,  Warner, 
Rosen blaff,  Hillelsohn,  &  Stelzer,  1978) 

(' Instructor  should  not  insist  on  his/her  instructional  sequences 
because  learners  often  prefer  different  sequences.  The  most 
meaningful  sequences  for  the  students  were  different  from  those 
used  by  the  instructors.  (Mager,  1961;  Mager  &  Clark,  1963) 

-  Learner  sequences:  "from  a  whole  to  a  more  complex  whole" 

-  Instructor  sequence:  "from  the  part  to  the  whole" 

H.  Prior  Knowledge 

HI      If  learners  have  prior  knowledge,  they  tend  to  select  more 

content  option  than  those  without  such  knowledge  (Nelson,  1985). 
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H2      Learners  are  likely  to  enter  a  learning  situation  WHEN  they  have 
a  significant  amount  of  relevant  knowledge.  (Mager  &  Clark, 
1963) 

H3      Use  Advance  Organizer  to  improve  the  effectiveness  of  learner 
control  (Mayer,  1976). 

H4      For  learners  with  a  low  domain  knowledge  base,  program  control 
is  superior  to  learner  control.  The  difference  diminishes  for 
those  with  a  high  domain  knowledge  base.  Learner's  self- 
judging  ability  is  contingent  on  the  level  of  prior  domain 
knowledge  (Lee  &  Lee,  1991). 

L  Motivation/Attitude/Feedback 

11  Provide  feedback  on  cumulative  performance  in  order  to 
increase  learner  control  effectiveness.  (Schloss,  Wisniewski,  & 
Cartwright,  1988). 

12  Provide  encouragement,  advice,  and  monitoring  strategies  in 
order  to  improve  effectiveness  of  learner  control  and  student 
'on-task'  time.  (Carrier,  Davidson,  &  Kalweit,  1986) 

13  In  order  to  increase  student  motivation  and  attitude  of  learning, 
use  learner  control  strategies  (Mager,  1961;  Newkirk,  1972). 

J.   Learning  Phases: 

Use  program  control  instead  of  learner  control  for  knowledge 
acquisition.  Meanwhile,  use  learner  control  instead  of  program  control 
in  the  knowledge  review  phase  (Lee  &  Lee,  1991). 

K.    Self-Assessment/Learning  Strategy 

Kl      Develop  learning  skills  in  order  to  justify  initiatives  for  creating 
learning  environment.  (Derry  &  Murphy,  1986) 

K2      Use  self-instruction  in  order  to  initiate,  plan,  organize,  monitor, 
and  regulate  own  learning  process.  (Weiner  &  Kluwe,  1987) 

K3      In  order  to  make  good  use  of  learner  control,  conduct  self- 
assessment.  However,  young  students  have  difficulty  assessing 
their  own  performance  or  their  en-route  comprehension. 
(Garhart  &  Hannafin,  1986) 

L.  Instructional/Learning  Systems 

If  one  understands  and  manages  learner  control  well,  one  will  feel 
comfortable  with  the  current  conception  of  intelligent  tutoring 
systems.  (Duchastel,  1986). 


Conclusion  &  Implication 

We  discuss  learner  control  as  one  of  the  main  issues  in  instructional 
management  and  the  importance  of  effectiveness  and  efficiency  in  the 
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context  of  instructional/learning  systems.  Instructional  prescriptions  of 
learner  control  are  developed  in  a  smorgasbord  fashion. 

Teachers  and  instructional  designers  may  have  the  chance  to  consider 
the  functional  relationship  between  learner  control  and  cognitive 
development  of  the  learner,  between  the  learner  control  and  types  of  learning 
(or  learning  domains),  and  between  learner  control  and  learner's  motivation. 
They  can  use  this  prescriptive  knowledge  base  of  learner  control  in  order  for 
their  students  to  maximize  the  effectiveness  and  efficiency  of  a  given 
learning  situation. 

At  this  point,  we  should  look  into  our  challenges  for  the  future.  All 
instruction  involves  some  learner  control.  Our  challenge  is  not  whether  or 
not  learner  control  should  be  used,  but  rather  how  to  maximize  the  learner's 
ability  to  use  the  learner  control  available.  Every  student  has  some  type  of 
internal  "how  to  learn"  model  which  directs  his/her  learner  control.  Thus, 
the  most  promising  area  for  future  exploration  is  to  help  learners  to  develop 
internal  adaptive  models  which  direct  their  use  of  appropriate  internal 
processing.  We  should  seek  to  identify  the  nature  of  learner  control  in  every 
instructional  situation  and  know  more  about  the  types  of  conscious  cognitive 
control  which  may  be  appropriate  for  different  outcomes  and  for  different 
displays.  Then,  the  student  who  has  the  most  choices  available  will  be  best 
able  to  maximize  the  effectiveness  and  efficiency  of  a  given  learning 
situation. 

From  a  different  view,  we  can  provide  three  suggestions  here  for 
tomorrow's  learner  control  research.  First,  outcomes  considered  within 
learner  control  research  should  include  the  development  of  effective  learning 
strategies,  the  continuing  motivation  to  learn,  and  long  range  achievement. 
Second,  explore  ways  in  which  advisement  can  be  offered  to  assist  individuals 
in  the  use  of  learner-controlled  features  and  regulation  of  their  own  learning. 
Third,  the  relationship  between  the  provision  of  learner  control  and  student 
perception  of  control  within  instruction  should  be  considered. 


9 

ERIC 


7  143 
1R3 


Figure  1,  Variables  of  Learner  Control 

Effectiveness 
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The  effects  of  different  forms  of  computer-mediated  feedback 
on  lesson  completion  time 

by  Roy  B.  Clariana 

Overview 

This  study  considered  the  effects  on  lesson  completion  rime  of  four  forms 
of  immediate  feedback.  Thirty-two  low-ability  eleventh  grade  students  were 
randomly  assigned  to  one  of  four  feedback  treatments.  Every  student  received  four 
social-studies  reading  passages  printed  on  regular  U.  S.  Letter  size  paper,  with  one 
passage  per  page.  Each  passage  averaged  350  words  in  length.  Eight  4-altemative 
multiple  choice  questions  were  presented  by  computer  with  each  passage,  with  one 
question  per  screen.  One  of  the  following  four  feedback  forms  was  provided  for 
each  treatment  condition.  Students  received  either  knowledge  of  correct  response 
feedback  (KCR),  which  provided  the  correct  alternative  after  the  students'  first 
attempt  or  KCR  with  second  try  (KCR  second  try),  which  allowed  the  student  to 
try  twice  before  the  correct  answer  was  provided  (Qariana,  Ross,  &  Morrison, 
1991;  Dempsey  &  Driscoll,  1989;  Noonan,  1984).  [Note,  if  students  are  correct  on 
their  first  try,  then  these  two  forms  are  not  different.  These  low-ability  students 
averaged  30%  correct  during  the  lesson,  suggesting  that  about  70%  of  the  items 
were  delivered  as  the  alternate  treatments.]  The  KCR  and  KCR  second  try 
conditions  were  completely  crossed  with  two  levels  of  context  termed  Full  and 
Focus  (Sassenrath  &  Yonge,  1969;  Sturges,  1969;  Winston  &  Kulhavy,  1988). 
Full-context  feedback  (Full)  presented  the  stem,  distractors,  and  the  correct 
alternative,  while  Focused-context  feedback  (Focus)  presented  only  the  stem  and 
correct  alternative,  the  distractors  were  not  shown. 

Summary  of  Achievement  Results 

Achievement  data  results  from  this  study  are  published  in  detail  in  Clariana 
(1990).  To  summarize  the  achievement  findings,  the  KCR  treatment  posttest  mean 
was  significantly  larger  than  the  KCR  second  try  treatment  posttest  mean,  EG  .28) 
=  4.561,  p  =  .041.  No  difference  was  shown  for  Full  and  Focus  contexts  or  for 
the  interaction  of  feedback  and  context. 

Feedback  and  Information  Processing 

The  simple  assumption  was  made  that  more  feedback  information  per  lesson 
would  translate  into  more  time  in  the  lesson.  Thus,  it  was  hypothesized  that  a 
significant  time  difference  (i.e  ,  an  interaction)  would  occur  between  the  KCR- 
Focus  group  (with  the  least  feedback  information)  and  the  KCR  second  try-Full 
group  (with  the  most  feedback  information).  Support  for  this  hypothesis  would 
provide  support  for  the  current  information  processing  model  of  feedback  function. 

Results 

The  time  data  were  analyzed  by  a  two  between  ANOVA  design  which 
included  feedback  (KCR,  KCR  second  try)  and  context  (Full,  Focus).  ANOVA 
results  of  total  lesson  time  data  did  not  reveal  the  hypothesized  interaction  (see 
Tables  1  and  2).  The  main  effect  for  feedback  [E(l,28)  =  1.933]  and  the  interaction 
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of  feedback  and  context  DE(1,2C,  *  0.081]  were  not  significant.  The  main  effect 
for  context  m&  significant,  E(l,28)  *  9.181,  p_  =  .005.  Examination  of  the  context 
treatment  means  showed  that,  unexpectedly,  the  students  took  more  time  to 
complete  the  Focus  treatments  Qc  =  2594  seconds)  than  to  complete  the  Full 
treatments  (x_  ■  1893  seconds). 


Table  1 

ANOVA  Summary  Table. 


source 

df 

as 

MS 

F 

E 

Treatment 

l 

827219.5 

827219.5 

1.933 

0.175 

Context 

l 

3929105.3 

3929105.3 

9.181 

0.O5* 

Treatment  x  Context 

l 

34650.3 

34650.3 

0.081 

0.778 

Error 

28 

11982665.1 

427952.3 

Table  2 

Time  spent  on  Items  and  on  Feedback,  in  seconds. 


Treatments  Items  Feedback  Feedback  Feedback  Lesson 

(screens)  zJ&M  -  2nd  try  Totals  Totals 

Full-KCR  1548  225  --  225  1773 

Full  -  KCR  2nd  try      1681  129  203  332  2013 

Focus  -KCR  2260  145  »  145  2405 

Focus  -  KCR  2nd  try   2374  112  297  409  2783 


Discussion 

The  expected  simple  linear  relationship  that  more  feedback  information  will 
mean  longer  lesson  time  was  npi  confirmed.  As  anticipated,  the  learners  did  spend 
more  time  in  the  KCR  second  try  treatment  (&  =  2398  seconds)  than  in  the  KCR 
treatment  (x  -  2089  seconds),  though  this  difference  was  not  significant.  Most  of 
the  time  difference  between  the  KCR  and  KCR  second  try  groups  is  explained  by 
the  time  spent  cn  second  tries  (&  -  250  seconds).  Interestingly,  the  learners  in  this 
study  who  knew  they  would  receive  a  second  try  (i.e.,  KCR  second  try  condition) 
spent  less  time  on  the  first  feedback  presentation  and  considerably  more  time  on  the 
second  feedback  presentation,  relative  to  the  one  try  (i.e.,  KCR)  group. 
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Contrary  to  expectation,  learners  spent  significantly  more  time  in  the  Focus 
treatment  &  =  2594  seconds)  than  in  the  Full  treatment  (*  -  1893  seconds).  Why 
should  the  Focus  condition,  with  less  information,  take  longer  to  complete?  First. 
the  feedback  form  mav  have  altered  how  the  students  used  the  feedback.  Learners 
in  the  Full  condition  spent  on  average  279  seconds  on  feedback  while  students  in 
the  Focus  condition  spent  on  average  277  seconds.  However,  examination  of  total 
feedback  time  means  shows  that  an  interaction  occurred  (see  Figure  1).  Learners  in 
the  Focus  condition,  when  given  only  one  try  (the  KCR  condition),  examined 
feedback  the  least,  as  might  be  expected.  However,  students  in  the  Focus  condition 
that  were  given  a  second  try  examined  the  feedback  that  was  given  after  their 
second  try  for  a  considerably  longer  time.  It  cannot  be  determined  in  this  study 
whether  this  additional  time  should  be  attributed  to  reading  feedback,  the  passage, 
or  both.  It  is  likely  that  the  learners  were  rereading  the  text  passage.  Giving 
learners  a  second  round  of  feedback  may  cause  them  to  reread  text  passages. 


500., 


KCR  KCR 

2nd  try 


Figure  1.       Line  graph  of  total  time  in  feedback  for  each  treatment. 


Was  reading  time  equally  distributed  over  each  question?  The  answer  is  no. 
Since  the  main  effect  Context  was  significant,  a  line  gnmh  of  the  average  time  for 
each  screen  for  Full  and  Focus  context  is  shown  in  Figure  2.  Note  the  four  reading 
passages  labelled  passage  1  through  passage  4,  There  are  36  distinct  time  points, 
one  for  each  screen.  The  first  screen  for  each  passage  was  a  brief  instruction  screen 
which  indicated  which  passage  the  student  should  read  to  answer  the  following 
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questions.  The  remaining  eight  points  for  each  passage  show  the  average  time  cor 
each  question,  which  includes  the  icedback  given  with  that  question. 


200-1 


time 
(seconds) 


Figure  2.       Average  time     jcreen  for  the  Full  and  Focus  Treatments. 


Several  trends  are  shown  in  Figure  2.  First,  the  students  spent  a  relatively 
long  amount  of  time  during  the  instruction  screen  (which  contained  one  short 
sentence)  md  then  more  time  with  the  first  question  in  the  sequence.  This  pattern 
was  repeated  with  each  succeeding  passage.  Probably,  the  students  read  the 
passage  initially  and  then  in  more  detail  after  the  initial  question.  Therefore,  ills 
possible  that  the  first  few  question  in  a  series  of  questions  mav  direct  or  influence 
the  sti  dent's  text  processing  approach  (i.e..  meta-level)  toward  that  text  passage.  If 
so,  pre  vious  adjunct  question  research  may  be  confounded  by  the  possibility  of 
question  order  effects.  Future  research  should  address  this  possibility.  Practically, 
this  suggests  that  Instructional  Designers  may  need  to  pay  close  attention  to  the  first 
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question  in  e  series  of  questions,  since  it  may  function  to  direct  the  student's 
processing  of  the  passage.  Also,  learners  in  all  four  treatment  conditions  spent 
more  time  with  the  first  passage  than  with  the  succeeding  passages  (passage  1 : 579 
seconds;  passage  2: 525  seconds;  passage  3: 473  seconds;  passage  4;  389 
seconds),  even  though  the  passages  were  similar  in  length  and  difficulty  (see 
Figures  2  and  3).  Thus,  the  initial  passage  mav  itself  serve  an  orienting  function. 

Second  and  more  importantly,  feedback  context  may  have  altered  how  the 
learners  used  supporting  materials  (see  Figure  3).  Note  that  learners  in  the  Full 
condition  spent  considerably  less  time  with  each  succeeding  passage,  while  learners 
in  the  Focus  condition  maintained  a  relatively  high  average  amount  of  time  on  the 
first  three  passages,  which  then  dropped  with  passage  four.  Learners  in  the  Focus 
treatment  could  be  described  as  "persisting"  throughout  the  lesson,  both  with 
individual  items  and  with  succeeding  passages. 


Figure  3.       Average  total  time  per  passage  for  the  Full  and  Focus  Treatments. 


Time  data  were  collected  both  during  the  lesson  and  during  the  achievement 
posttest.  Interestingly,  a  high  correlation  was  observed  between  the  lesson  and 
posttest  completion  limes  [R=0.60,  E(l,31)=16.866,  p.=0.0003].  Two  alternate 
explanations  occur.  One,  learners  have  a  characteristic  way  or  rate  of  responding 
during  computer  instruction,  or  two,  the  lesson  treatments  established  an  episodic, 
conversational,  or  rhythmic  response  pattern  that  carried  over  to  the  posttest.  In 
this  study,  probably  both  are  occurring.  Either  alternative  has  serious  implications 
for  research  involving  time  or  rate  variables',  since  confounding  is  likely  to  occur  in 
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studies  that  do  not  control  these  variables.  Future  research  should  address  these 
alternative  models. 

In  summary,  the  significant  time  difference  observed  for  Focus  over  Full 
feedback  is  due  to  increased  persistance  both  with  each  item  in  a  series  and  with 
each  succeeding  passage,  especially  with  later  passages  in  the  sequence  (see 
Figures  2  and  3).  Does  less  information  in  the  Focus  condition  cause  the  student  to 
spend  more  time  with  the  reading  passage  relative  to  the  Full  condition? 
Alternately,  persistence  may  be  an  individual  learner  characteristic,  thus  random 
assignment  in  this  study  failed  with  most  "persisters"  falling  in  the  Focus  group. 

If  feedback  form  does  in  fact  alter  learners'  responses  to  text  passages,  then 
current  information  processing  models  that  view  feedback  from  a  "quantity"  of 
information  viewpoint  seem  inadequate  to  explain  this  finding.  Different  forms  of 
feedback  may  qualitatively  alter  a  learner's  approach  to  supporting  lesson  material. 
Additional  research  should  consider  the  possibility  of  qualitative  as  well  as 
quantitative  effects  of  feedback. 
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Abstract 


The  present  study  investigated  the  effects  of  using  different  forms  of  material. 
The  basic  design  consisted  of  two  conditions  of  instructional  support  (text  and 
questions  vs.  questions  only),  two  testings  (immediate  vs.  retention),  five 
levels  of  similarity  between  lesson  and  posttest  questions,  and  five  feedback 
conditions:    Knowledge  of  Correct  Response  (KCR),  delayed  KCR,  Answer  Until 
Correct  (AUC),  questions  only  (no  feedback),  and  no  questions.      Results  showed 
significant  benefits  for  feedback  over  no-feedback,  with  AUC  becoming  more 
advantageous  and  delayed  feedback  less  so  as  lesson-posttest  question 
similarity  decreased.     Also,  with  decreased  question  similarity  and  the 
availability  of  supporting  text,  overall  feedback  effects  tended  to  decrease.  The 
results  are  discussed  in  terms  of  the  information  processing  effects  of  the 
different  feedback  forms,  a  factor  that  CBI  designers  often  fail  to  exploit  in 
planning    feedback  conditions. 
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The  Effects  of  Different  Feedback  Strategies  Using  Computer-Administered 
Multiple-Choice  Questions  as  Instruction 

The  use  of  feedback  is  a  critically  important  and  often  neglected 
attribute  in  computer-based  instruction  (CBI),     feedback  promotes  learning  by 
providing  students  with  information  about  their  responses.     Through  its 
interactive  capabilities,  CBI  increases  the  range  of  feedback  strategics  that 
can  be  efficiently  achieved,     Specifically,  when  incorporated  in  multiple- 
choice  testing,  three  common  forms  include  (a)  knowledge  of  response 
feedback  (KOR  or  KR),  which  indicates  that  the  learner's  response  was  correct 
or  incorrect;  (b)  knowledge  of  correct  response  feedback  (KCR),  which 
identifies  the  correct  response;  and  (c)  elaborative  feedback,  which  identifies 
the  correct  response  while  providing  additional  explanations  (Merrill,  1985), 

As  would  be  expected,  these  forms  of  feedback  may  not  be  equally 
effective,    Several  studies  have  shown  KCR  to  be  superior  to  KOR,  and  KOR  to  be 
superior  to  no  feedback  (Gilman,  1969;  Kulhavy,  1977;  Travcrs,  Van  Wagcncn, 
Haygood,  &  McCormick,  1964;  Waldrop,  Justin,  &  Adams,  1986).       However,  based 
on  his  own  research  and  a  meta-analysis  of  studies,  Schimmel  (1983;  1986) 
concluded  that  this  hierarchy  of  immediate  feedback  types  is  not  so  well 
established.     Evidence  also  suggests  that  elaborative  forms  of  feedback  often 
produce  no  significant  improvement  over  KCR,  but  require  a  considerable 
development  and  implementation  cost  (Merrill,  1985,  1987;  Spock,  19*.  7), 
Despite  years  of  research,  the  types  of  situations  in  which  different  feedback 
forms  tend  to  operate  most  effectively  are  still  not  understood.    Part  of  the 
reason  nr  j  be  a  failure  to  account  adequately  for  the  influences  on  results  of 
task  and  learner  characteristics  as  well  as  the  cognitive  (as  opposed  to 
behavioral)  impact  of  the  different  feedback  treatments  employed  (see 
Hannafin  &  Rieber,  1989).    Kulhavy  &  Stock  (1989)  further  attribute  the  lack  of 
understanding  of  feedback  effects  to  the  reinforcement  emphasis  of  the 
operant  conditioning  paradigm   that  predominated   research   and   theory  for 
many  years.     In  their  current  model,  they  stress  the  cognitive  implications  of 
feedback  effects  on  information  processing,  while  indicating  that  systematic 
research   illuminating  such  effects  has  been  minimal. 

Usually,  feedback  is  provided  to  the  learner  after  one  response, 
However,  using  CBI,  a  learner  may  easily  be  allowed  a  second  try  with  an  item 
(Dempsey  &  Driscoll,  1989;  Noonan,  1984)  or  may  be  required  to  continue  to 
respond  until  the  correct  answer  is  selected  (Pressey,  1926,  1950),    The  latter 
orientation  is  conventionally   labeled   answer-until-correct   (AUC)  feedback. 
Allowing  unassisted  multiple  response  tries  has  considerable  intuitive  appeal. 
AUC  may  engage  learners  in  additional  active  processing  following  errors  and 
also  ensures  that  the  last  response  is  a  correct  one,  a  prin  ciple  espoused  over 
half  a  century  ago  in  the  contiguity  theory  of  Edwin  Guthrie  (1935). 
Unfortunately,  there  have  been  relatively  few  controlled  empirical  studies  to 
test  this  interpretation  or  whether,  in  general,  allowing  one  response  or 
requiring  many  responses  to  an  item  is  more  effective  (Dempsey  &  Driscoll, 
1989;  Noonan,  1984),    On  the  one  hand,  providing  the  correct  answer  after  only 
one  response  may  "short-circuit"  learning  (Schimmel,   1986).  Alternatively, 
requiring  a  learner  to  answer  until  correct  may  be  frustrating  (Dick  &  Latta* 
1970). 
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Feedback  timing  is  another  variable  of  interest  (Hannafin  &  Reiber, 
1989;  Kulhavy  &  Stock,  1989).    Feedback  may  be  provided  immediately  after  the 
learner's  response  or  it  may  be  delayed  for  either  a  set  period  of  time  or  set 
number  of  responses,  such  as  at  the  end  of  a  test.    From  a  recent  meta-analysis, 
Kulik  and  Kulik  (1988)  concluded  that  immediate  feedback  was  best  for  most 
learning  situations,  but  delayed  feedback  was  superior  in  "test-acquisition" 
studies,  i.e.,  learning  situations  in  which  test  questions  arc  used  as  the 
instruction.     Two  interpretations  are  most  commonly  used  to  explain  test- 
acquisition  benefits,     One  is  termed  the  interference  perseveration  hypothesis 
(Kulhavy  &  Anderson,  1972).    This  view  holds  that  an  incorrect  response 
proactively  interferes  with  an   immediately  provided   correct  response. 
Delaying  the  presentation  of  feedback  allows  learners  time  to  forget  their 
initial  responses,  thereby  reducing  proactive  interference  effects.  However, 
if  such  is  the  case  proactive  interference  should  occur  not  only  in  test- 
acquisition  studies  but  in  any  manipulation  of  immediate  and  delayed 
feedback.     A  possible  explanation  concerns  the  instructional  support  other 
than  embedded  questions  that  most  lessons  provide,     Such  support  may  consist, 
for  example,  of  reading  passages,  pictures,  outlines,  overviews,  or  video-clips, 
This  support  may  serve  to  make  the  the  material  more  memorable  and  thus 
more  resistant  to   proactive  interference. 

The  second  interpretation  is  based  on  the  rationale  that  delayed 
feedback  repeats  the  item  presentation  at  the  end  of  the  lesson,  thereby 
providing  twice  as  much  exposure  than  does  immediate  feedback  (Kulik  & 
Kulik,  1988),    But  if  other  forms  of  instructional  support  are  included,  such  as 
the  addition  of  a  reading  passage,  the  effects  of  the  double  exposure  are  likely 
to  be  mitigated,    Delayed  and  immediate  feedback  would  then  produce 
comparable  results,      Both  the  interference  and  frequency-of-feedback  views 
appear  to  provide  valid  explanations  of  feedback-timing  effects  and  both  are 
supported  by  research  (More,  1969;  Newman,  Williams,  &  Hillcr,  1974;  Peeck  & 
Tillema,  1978;  Suber  &  Anderson,  1975),    The  role  of  text  as  instructional 
support  for  questions  (i,e,,  test-acquisition   vs,  text-with-questions  effects), 
however,  has  not  been  adequately  investigated. 

The  literature  on  feedback  also  leaves  questions  unanswered  regarding 
the  relationship  of  lesson  questions  to  posttest  questions,     Often,  posttest 
questions  are  identical  in  form  and  wording  to  the  lesson  questions,  a  rote 
recognition  condition  that  substantially  restricts  the  degree  to  which  results 
can  be  generalized  to  typical  learning  situations,     Bormuth,  Manning,  Carr  and 
Pearson  (1970)  demonstrated  experimentally  how  posttest  questions  could  be 
adapted   from  instructional   reading  passages  to  measure  comprehension 
learning  by  transforming  and  paraphrasing  text  (also  see  Anderson,  1972). 
Using  their  approach,  the  present  study  was  designed  to  compare  the  effects 
on  learning  of  three  types  of  feedback  strategies  (KCR,  AUC,  and  delayed) 
applied  to  five  levels  of  lesson  questions  differing  in  degree  of  relatcdncss  to 
posttest    questions.  An  additional  variable  was  whether  feedback  treatments 
were  presented  with  associated  text  passages  or  with  no  text  (i.e.,  test- 
acquisition,     Subjects  were  low-ability  high  school  students  enrolled  in  a 
summer    preparatory    program  in  science.    The   following   hypotheses  were 
tested, 

1,        The  provision  of  feedback  vs,  no  feedback  would  improve  learning 
across  all  feedback  strategies  and  questioning  levels. 
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2  AUC  feedback  would  become  increasingly  effective  relative  to  other 
feedback  forms  as  question  level  (disparity  between  lesson  and  posttest 
questions)  increased,  due  to  providing  additional  processing 
opportunities  and  review  of  the  information  to  be  learned. 

3  The  provision  of  text  would  become  more  facilitative  as  question  level 
increased  due  to  furthering  understanding  of  the  material  through  the 
provision  of  additional  descilptirns  and  explanations. 

Method 

Subjects  consisted  of  100  eleventh  grade  students  enrolled  in  a  five- 
week  CBI  summer  enrichment  program  sponsored  by  Memphis  Partners 
Incorporated.     Memphis  Partners  selects  students  considered  to  be  at-risk  Irom 
all  schools  in  the  metiopolitan  area.     All  subjects  voluntarily  participated  in 
.nis  study  which  was  described  as  an  American  College  Test  (ACT)  preparation 
course.    All  subjects  were  black;  their  median  age  was  17.    To  qua  ify  for  the 
program,  they  needed  to  (a)  be  entering  the  12th  grade  in  the  fall;  (b)  have 
low  ACT  scores  (between  10  and  15);  and  (c)  be  described  by  their  guidance 
counselors  and  teachers  in  a  written  recommendation  as  having  academic 
potential  for  college,  despite  their  low  standardized  achievement  scores. 

Subjects  were  randomly  assigned  to  one  of  10  treatment  groups 
consisting  of  five  feedback  conditions  (KCR,  AUC,  delayed  KCR,  questions  only, 
and  no  questions)  crossed  with  two  conditions  of  instructional  support  (text 
and  no-text).     Within-subjects  factors  consisted  of  five  question  levels 
(verbatim-identical,    inferential-identical,  inferential-transformed, 
inferential-paraphrased,    and   transformed-paraphrased),    and    two  testings 
(immediate  and  retention).    The  analytical  design  thus  consisted  of  a 
5(feedback)  x  2(instructional  support)  x  5(question  level)  x  2(lesling)  mixed 
analysis  of  variance  (ANOVA). 

Prior  to  the  start  of  the  instructional  phase  of  the  study,  all  subjects 
were  administered  the  ACT  Natural  Science  Reading  Test  and  the  Nflson-Denny 
Reading  CjfflUttgten&iiS  Tpst  F"rm  E  t0  assess  trcatmcnt  8rouP  equivalence. 
The  former  test  is  a  measure  of  science  knowledge,  and  the  latter  is  a  measure 
of  reading  ability.     Analyses  of  pretest  scores,  using  a  5(fcedback)  x  2(support) 
ANOVA,  indicated  no  significant  differences  between  treatment  groups  on 
cither  measure, 

Tngtrurtjftpal    and   Assessment  Materials 

Text  passages.    The  reading  materials  were  adopted  from  the  ACT 
National  Science  Reading  Test.  8223c,    They  included  four  ».c>  passages 
entitled  "Solids/  "Genetics,"  "Compressed  Gas,"  and  "Trojan  Asteroids."  The 
average  number  of  words  per  passage  was  350.    All  passages  were  presented  in 
print  form  to  allow  subjects  continual  access  to  them  during  the  lesson  and  to 
create   i  more  realistic  learning  situation.     Readability  of  the  passages,  using 
Dalc-Chall  (1948)  and  Flesch  (1948)  procedures,  ranged  from  10th  grade  to 
-jollege. 

T.esson  and  naffest  questions.    For  each  passage,  10  lesson  questions 
were  constructed.     Because  the  instructional  orientation  of  the  ACT  passages 
and  achievement  test  (ACT  samnle  Test  8223C)  emphasized  inferential  learning 
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(i.e.,  reasoning  from  the  passage  to  soivc  a  problem  or  application),  it  was 
decided  to  make  8  out  of  each  10  (80%)  lesson  questions  inferential  and  the 
remaining  two  questions  (20%)  verbatim.     Inferential  questions  required 
going  beyond  the  specific  text  information  to  formulate  an  idea  or  concept  not 
explicitly  stated.    Verbatim  questions  repeated  the  text  passage  word  for  word. 

Each  of  40  lesson  questions  was  made  parallel  to  an  existing  verbatim  or 
inferential  posttest  question  adapted  from  the  ACT  Sample  Test  8223c.  Posttest 
questions  were  then  varied  in  form  on  a  random  basis  for  the  purpose  of 
assessing,  different  levels  of  learning.     For  the  inferential  questions  a 
2(transformation)  x  2(paraphrase)  design  matrix  was  used  to  achieve  these 
levels     As  will  be  described  below,  one  factor  was  whether  a  structural 
transformation  or  the  original  form  of  the  corresponding  lesson  question  was 
used;  the  other  factor  was  whether  paraphrased  wording  or  original  wording 
was  used.    Specifically,  transformed  posttest  questions  reversed  the  stem  and 
the  answer  from  the  corresponding  lesson  question.     To  illustrate  using  a 
simple  example,  the  question,  "The  capital  of  Arkansas  is:    (a)  Little  <Rock  (b) 
Memphis  (c)  Dallas"  would  be  transformed  to  read:    "Little  Rock  is  the  capital  o, 
(a)  Arkansas  (b)  Tennessee  (c)  Texas."    Paraphrased,  questions  were 
constructed  to  maintain  the  same  structure  and     meaning  as  corresponding 
lesson  questions,  but  using  different  words  or  phrasing. 

These  manipulations  resulted  in  five  levels  of  posttest  questions.    Table  1 
illustrates  the  five  question  forms  in  relation  to  a  lesson  text  segment 
containing  tested  content.     Both  lesson  questions  and  posttest  questions  were 
administered  on  a  WICAT  System  300  microcomputer  with  30  student  stations. 
The  five  forms  are  summarized  below: 

1.  Yerbat'm-Identical  (VI)  tested  verbatim  learning  using  the  same 
wording  as  the  original  text  and  the  lesson  questions. 

2.  Inferential-Identical   (II)  tested  inferential  learning  using  similar 
wording  as  the  text  and  the  identical  wording  as  the  lesson  questions. 

3.  Inferential-Transformed   (IT)  tested  infer  itial  learning  using 
similar  wording  as  the  text,  but  the  question  answer  and  stem  were  reversed 
relative  to  lesson  questions. 

4.  Inferential-Paraphrased  (IP)  tested  inferential  learning  using 
different  words  and  phrasing  relative  to  the  text  and  the  lesson  questions. 

5.  Transfonned-Par  aliras_eji.  (TP)  tested  inferential  learning  using 
both   a  transformed   structure   and  paraphrasing. 


Insert  Table  1  about  here 


Reliability  of  the  postfst  and  lesson  questions  was  assessed  employing 
94  high  school  students  (47  lor  each  question  set).     Split-half  reliability,  using 
the  Spearman-Brown  formula,  was  ,<56  for  the  posttest  set  and  .85  for  the  lesson 
set. 

Text  and  Feedback  Treatments 

Manipulation  of  the  text  variable  involved  either  providing  or  not 
providing  the  passages  to  read  during  the  lesson.     The  feedback  variable 
consisted  of  three  feedback  forms  and  two  no-feedback  control  conditions 
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(questions  without  feedback  and  no  questions).     When  presented  without  text, 
*he  no-questions  variation  represented  a  pure  control  condition  in  which 
students  were  administered  the  two  posttests  without  receiving  prior 
instruction.     Each  of  the  three  feedback  conditions  is  described  below. 

1     Knowledge  of  correct  response  (KCR),    This  condition,  which  was 
patterned  after  that  used  by  Dempsey  '1988;  also  Tail,  Hartley,  &  Anderson, 
1973),  informed  the  learner  of  the  correct  answer  after  each  response. 
Specifically,  following  a  correct  response,  the  word  "RIGHT"  was  displayed  at 
the  bottom  of  the  computer  screen.     Following  an  incorrect  response,  the  word 
"WRONG"  was  displayed  with  the  correct  answer  designated  by  an  arrow.  The 
learner  was  instructed  to  type  the  letter  of  the  correct  answer  to  continue. 

2L  Answer  until  correct  (AUC).    AUC,  based  on  Dempsey  (1988),  provided 
the  sami  feedback  as  KCR  following  correct  responses.     However,  following  the 
first  incorrect  answer  to  a  given  question,  the  prompt,  "NO  TRY  AGAIN  "  was 
displayed  at  the  bottom  of  the  screen.    The  learner  then  made  a  second  try, 
which  if  correct  was  followed  by  the  usual  "RIGHT,"  and  if  incorrect  was 
followed  by  "WRONG"  along  with  the  instruction  to  type  in  the  letter  of  the 
correct  response  (as  designated  by  the  wow).    Thus  AUC  was  identical  to  KCR, 
except  for  the  second  try  given  following  an  initial  error  response. 

Delayed   feedback.    This  condition  provided  KCR-typc  feedback  at  the 
conclusion  of  all  four  lesson  sections  by  individually  presenting  the  40 
questions  in  original  order,  with  the  correct  answer  for  each  designated  by  an 
arrow.     Separate  from  this  concluding  feedback  display,  an  additional  design 
consideration  was  whethor  to  provide  any  immediate  feedback  to  indicate  the 
accuracy  of  responses.     Given  the  difficulty  and  technical  nature  of  the 
subject  matter,     we  reasoned  that  the  absence  of  such  information  would  be 
frustrating  to  learner;  and  unrealistic  relative  to  what  would  probably  be 
done  by  most  designers  in  practice.    Accordingly,  we  decided  on  a  "middle 
ground"  approach  in  which  immediate  feedback  was  provided,  but  the  message 
was  downgraded  in  "load"  (information  density)  to  KOR  (as  opposed  to  KCk); 
i.e.,  simply  indicating  that  the  answer  was  "RIGHT"  or  "WRONG,"  without 
designating  the  correct  answer.     Typically,  the  time  delay  from  the  learner's 
first  response  to  the  item  to  the  delayed  KCR  was  about  30  minutes. 

Procedure 

The  summer  preparation  program  continued  for  five  weeks  during 
students'  school  vacation.     On  selected  weeks  students  attended  experimental 
sessions,  referred  to  as  an  "ACT  prep  course,"  for  one  hour  at  a  convenient 
time  during  the  day  or  night.     Prior  to  their  participation  in  the  experiment, 
they  had  been  administered  the  ACT  Natural  Science  Reading  Test  to  determine 
group  equivalence.     During  Week  1  of  the  research  period,  they  were 
administered  the  Nelson-Denny   Reading   Comprchcnsiou   Test,  as  an  additional 
measure   of  equivalence. 

During  Week  2,  subjects  participated  in  the  treatment  phase,  receiving 
instruction  appropriate  to  their  assigned  condition.     Lesson  questions  were 
administered  in  blocks  of  10  by  computer  in  all  conditions  except  the  no- 
questions  (posttcst-only)  treatment.      Supporting  text,  where  prescribed,  was 
available  at  the  learning  station  in  print  booklet  form.     The  qucstions-and- 
text  and  no-text  treatments  were  conducted  on  alternate  days  to  avoid  subjects 
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becoming  aware  of  the  alternative  experimental  condition.     Students  in  the 
questions-and-text  condition  were  allowed  to  use  the  text  in  any  v 
wished     They  were  not  gwen  additional  instructions  relative  to  .t 
condition  except  for  one  sentence  at  the  beginning  of  each  block  o.  .estions, 
indicating  that  they  should  read  a  particular  section  "to  help  answer  the 
questions"    They  were  then  left  on  their  own  to  read  and  reference  the  text 
whenever  and  for  as  much  time  as  they  wanted.    Observation  of  subjects 
reflected  use  of  a  variety  of  strategies,  including  reading  the  text  first  and 
then  answering  the  lesson  questions,  reading  the  text  as  questions  were 
answered,  and/or  referencing  parts  of  the  text  following  the  completion  of 
different  questions.    No  text  material  was  provided  during  the  posttest  or 
delayed  posttest. 

After  subjects  completed  the  assigned  treatment,  they  were  given  a  10- 
minute  break  followed  by  the  administration  of  the  40-item  posttest.  Subjects 
could  spend  as  much  time  as  they  needed  to  complete  the  instructional  phase; 
most  finished  in  45  minutes  to  1  hour.    Two  weeks  later,  they  were 
readministered  the  posttest  unannounced  to  assess  retention.     The  text  portions 
were  not  available  during  either  testing. 

Results 

The  analyses  of  achievement  scores  used  a  5  x  2  x  2  mixed  ANOVA  on 
cacn  question  level.     Between-subjects  factors  were  five  feedback  conditions 
(KCR    AUC,  delayed,  no-feedback,  no-questions)  and  two  instructional  support 
conditions  (text  vs.  no-text).     The  within-subjects  factors  were  five  question 
levels  (VI,  II,  IT,  IP,  and  TP)  and  two  testings  (posttest  and  retention  test). 
Means  for  all  conditions  are  shown  in  Table  2. 


Insert  Table  2  about  here 


The  ANOVA  yielded  s  gnificant  main  effects  due  to  feedback,  E  (4,  90)  = 
13.96,  &,  <  .001,  MSe  =  5.99;  and  question  level,  E  (4,  360)  =  135.32,  p.  <  .001,  MSe  - 
.92.    Each  of  these  effects  was  qualified  by  significant  interactions.  Two-way 
interactions  that  reached  significance  were  feedback  x  question,  E  (16,  360)  - 
13.50,  £,  <  .001,  MSe  =  .92;  support  x  question,  E  (4,  360)  =  2.46,  p.  <  .05,  MSe  -  .92; 
and  question  x  testing,  E(4,  360)  =  28.60,  fi.  <  .001,  MSe  =  1.19.    Significant  three- 
way  interactions  were  feedback  x  question  x  testing,  E  (16,  360)  =  4.06,  p.  <  .001, 
MSe  =  1.19;  and  support  x  question  x  testing,  E(4,  360)  =  5.20,  &  <  .001;  MSe  =  1.19. 
Further,  the  four-way  interaction  also  reached  significance,  E  (16,  370)  =  2.29, 
p<  .003,  MSe=  1.19. 

Interpretation  of  the  latter  interaction,  which  qualifies  all  other 
effects,  is  obviously  complicated  by  the  four  factors  and  100  means  it 
encompasses.     Given  that  every  interaction  involving  questions  x  testing  was 
significant,  it  seemed  appropriate  for  simplifying  the  interpretation  of 
interaction  patterns  to  conduct,  as  follow-up  analyses,  separate  feedback  x 
support  x  question  ANOVAs  for  each  test  (immediate  and  retention).    Due  to  the 
large  number  of  factors  involved  in  these  analyses,  the  .01  level  was  used  in 
judging  significance.     Results  from  each  analysis  are  summarized  in  Table  3 
and  reported  in  the  sections  below. 
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Immediate  Test  .        .  .  >  mn 

As  shown  in  Table  3,  the  feedback  (p.  <  .001)  and  question  level  (p.  <  .001) 
main  effects,  but  not  the  support  main  effect,  were  significant.  Significant 
interactions  were  feedback  x  question  level  (&  <  .001),  and  support  by  question 
level  (u  <  .01). 

The  feedback  main  effect  was  further  analyzed  via  Tukey  foliow-up 
comparisons  of  the  five  overall  treatment  means.     Results  indicated  that  all 
three  feedback  strategies,  KCR  (M=4.7),  AUC  (M  =  4.7),  and  delayed  (M=4.5),  were 
superior  (ft  <  .05)  to  both  the  no-feedback  (M.=3.2)  and  no-questions  (Mr=3.0) 
control  strategies.     Similarly,  Tukey  follow-up  tests  of  the  question  level  main 
effect  showed  means  on  the  two  idjailkai  question  forms,  verbatim-identical 
(M.=4.1)  and  inferential-identical  (M.=5.2),  to  surpass  (p_  <  .05)  the  means  on  the 
three  reworded   question  levels,  inferential-paraphrased   (M_=3.5),  inferential- 
transformed  (M.=3.4),  and  transformed-paraphrased  (M=3.1). 

The  significant  feedback  x  question  interaction  (p_  <  .01)  reflected  a 
general  pattern  for  larger  differences  favoring  feedback  means  over  control 
means  to  occur  on  identical  questions  (VI  and  II)  than  on  reworded  questions 
(sec  Figure  1).     Followup  analyses  involved  comparing  the  five  feedback 
means,  using  a  Tukey  test,  for  each  type  of  question.    The  .01  level  of 
significance  was  used  to  reduce  the  overall  Type  I  error  rate.  Findings 
indicated  that  on  both  verbatim-identical   and   inferential-identical  questions, 
each  of  the  three  feedback  groups  (KCR,  AUC,  and  delayed)  significantly 
surpassed  each  of  the  control  groups  (no-feedback  and  no-questions).  On 
inferential-paraphrased  questions,  the  only   significant  difference   was  that 
AUC  surpassed  no-questions.     No  differences  were  found  on  either  inferential- 
transformed   or   transformed-paraphrased  questions,   although   on   the  former 
measure,  the  differences  favoring  the  highest  group,  AUC,  over  both  of  the 
control  groups,  approached  significance  (.01  <  c's  <  .05). 


Insert  Figure  1  about  here 


Followup  analyses  of  the  support  x  question  'cvel  interaction  (p.  <  .01) 
consisted  of  comparing  the  tcxt-and-question  mean  to  the  questions-only 
mean  on  each  of  the  five  question  levels  (see  immediate  test  column  means  on 
Table  2).    Multiple  1  tests,  each  using  a  .01  significance  level,  were  used. 
Findings  revealed  that  the  only  significant  effect  occurred  on  verbatim- 
identical  questions,  with  text-and-questions  (M.-5.5)   surpassing  questions-only 
(M=4.6). 
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Retention  Test  , 

The  same  main  effects  and  interactions  that  were  significant  on  the 
immediate  test  were  also  significant  on  the  retention  test,  with  the  exception 
of  the  support  x  questions  interaction  (p.  >  .01;  see  Table  2).    The  feedback  main 
effect  (p.  <  .001)  was  further  analyzed  by  Tukey  tests.    As  occurred  for  the 
immediate  test,  KCR  (M=3.9),  AUC  (M=4.0),  and  delayed  (M=38)  feedback  each 
surpassed  no-feedback  (M.-3.1)  and  no-questions  (M_=3.0).    Followup  analyses  of 
the  question-level  main  effect  (p_  <  .001)  indicated  that  scores  on  verbatim- 
identical  (M=4.2)  and  inferential-identical  (M.=4.3)  questions  were  higher  than 
those  on  transformed-paraphrased  (M=2.9),   infercntial-transfjrmcd  (M.=3.2), 
and   inferential-paraphrased   (M=3.3)  questions.      The  transformed-paraphrased 
mean  was  significantly  lower  than  each  of  the  other  question-level  means. 

The  significant  feedback  x  questions  interaction  (e.  <  .001)  was  further 
analyzed  by  comparing  the  five  feedback  means,  using  a  Tukey  test,  for  each 
type  of  question  (alpha  =  .01).    The  interaction  is  graphically  displayed  in 
Figure  2.     Findings  indicated  that  on  verbatim-identical  questions,  the  two 
highest  groups,  delayed  and  KCR,  surpassed  the  two  lowest  groups,  no- 
questions  and  no-feedback;  AUC  did  not  differ  from  any  other  groups.  On 
inferential-identical  questions,  all  three  feedback  groups  surpassed  both 
control   groups.      On  inferential-transformed  questions,  no  significant 
differences  occurred;  the  largest  difference,  that  favoring  AUC  over  no- 
questions,  approached  significance  at  the  .01  level  (p_  <  05).    On  inferential- 
paraphrased  questions,  the  only  significant  difference  was  that  AUC  surpassed 
no-questions.      On   transformed-paraphrased   questions,   no  differences 
occurred.     The  overall  pattern  revealed  from  these  comparisons  is  similar  to 
but  not  as  strong  as  that  for  the  immediate  test,  showing  (a)  larger  differences 
favoring  the  feedback  conditions  over  the  control  conditions  on  the  two 
identical  question  types  (VI  and  II)  than  on  the  three  reworded  question  types 
(IT,  IP,  and  TP),  and  (b)  a  tendency  for  AUC  effects  to  be  more  positive  relative 
to  the  other  feedback  treatments  on  reworded  than  on  identical  question  types. 


Lesson  completion  times  were  analyzed  for  subjects  in  the  three 
feedback  conditions  using  a  3(feedback)  x  2(tcxt)  ANOVA.    The  text  main  effect, 
F_  (1,52)  as  38.9,  p_  <  .001  was  significant,  confirming  the  expected  longer 
completion  times  for  subjects  who  received  text  (M.  =  57.4  min.)  than  for  those 
who  did  not  (28.9  min.).    The  feedback  main  effect  approached  significance,  F 
(2,54)  -  2.89,  p_  <  .06,  with  the  ordering  of  means  from  highest  to  lowest  being 
delayed  (M  «  50.5),  AUC  (M  =  41.6),  and  KCR  (M  =  37.4). 


The  results  of  this  study  supported  the  hypothesized  benefits  for 
learning  of  providing  response  feedback  on  embedded  lesson  questions.  To 
most  educators,  this  result  would  hardly  be  viewed  as  surprising.  The 
effectiveness  of  feedback  is  a  basic  tenet  of  instructional  theory  that  has  been 
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demonstrated  countless  times  by  researchers  beginning  with  the  classic 
verbal  learning  studies  by  Thorndike  (1931)  on  the  effects  of  saying  "Right" 
or  "Wrong"  following  a  subject's  response.     Frequent  and  consistent  use  of 
feedback  is  also  strongly  promoted  in  today's  textbooks  on  teaching  and 
educational  psychology  (e.g.,  W^olfolk,  1990,  pp.  543-545;  Slavin,  1988,  pp.  383- 
387),    But,  while  the  benefits    "  feedback  in  general  might  be  taken  for 
granted,  uncertainty  still  exists  regarding  how  to  select  and  optimize  uses  of 
different  forms  of  feedback  depending  on  characteristics  of  students  and  the 
learning  situation. 

As  suggested  from  the  present  results,  one  important  factor  influencing 
feedback  effects  is  the  type  of  questioning  employed,  a  variable  that  has 
typically  not  been  controlled  in  previous  studies.    Had  only  one  level  of 
questioning  been  used,  our  findings  would  have  been  directly  dependent  on 
the  particular  level  selected.     Given  the  broader  perspective  obtained  by 
manipulating  five  questioning  levels,  we  were  able  to  detect  several  basic 
trends.    One  was  for  feedback  benefits  to  decrease  as  the  similarity  of  posttest 
questions  to  corresponding  lesson  questions  decreased.     In  other  words,  larger 
feedback  effects  occurred  on  the  "identical"  items  than  on  the  reworded  ones. 
Another,  in  support  of  Hypothesis  2,  was  for  the  relative  benefits  of  AUC 
feedback  to  increase  as  posttest  question  similarity  decreased.     Third,  feedback 
effects  relative  to  the  control  conditions  tended  to  be  greater  without  text  than 
with  text, 

Better  understanding  of  these  outcomes  can  be  obtained  by  analyzing 
the  nature  of  the  instructional  support  provided  by  the  different  feedback 
conditions.     As  suggested  here  and  in  previous  studies  involving  identical 
lesson  and  posttest  questions  (Kulhavy  1977;  Kulhavy  &  Anderson  1972;  Smith 
1988),  the  most  direct  benefit  of  KCR-type  feedback  (whether  immediate  or 
delayed)  is  informing  learners  of  the  correct  answers  to  lesson  questions. 
Thus,  even  when  the  level  of  learning  does  not  extend  beyond  rote 
memorization,  the  benefit  should  be  an  increased  ability  to  reconstruct  those 
associations  and  identify  the  answers  when  the  same  questions  appear  again 
on  a  lesson  posttest.    Looking  again  at  Figure  1,  that  effect  is  reflected  by  the 
three  feedback  groups'  greater  superiorly  over  two  control  groups  in  the  two 
conditions  where  posttest  questions  wu*e  exact  replications  of  lesson  questions 
(verbatim-identical    and  inferential-identical). 

It  has  further  been  proposed  that  for  strengthening  associations 
between  questions  and  correct  answers,  delayed  feedback  is  especially 
advantageous  by  providing  a  second  exposure  to  the  item  presentation  at  the 
end  of  the  lesson  (Kulik  &  Kulik,  1988)  and  by  reducing  proactive  interference 
(Kulhavy  &  Anderson,  1972),    Consistent  with  the  emphasis  of  these 
explanations  on  rote-learning  processes  (i.e.,  connecting  specific   answers  to 
associated  questions),  dclayed-feedback  effects  have  primarily  bee,    found  in 
situations  involving  identical  lesson  and  criterion  test  items  (e.g.,  Kulik  & 
Kulik,  1988;  Sturgis,  1978;  Suber  &  Anderson,  1975).    Similarly,  on  both  the 
immediate  and  retention  tests  in  the  present  study,  dclayed-feedback  was 
dircctionally  higher  than   all   other  feedback  conditions  on  verbatim-identical 
questions,  and  was  significantly  higher  than  the  control  conditions  on 
verbatim-identical   and   inferential-identical  questions  only.      Looking  at 
Figures  1  and  2,  the  effectiveness  of  delayed  feedback  relative  to  the  other 
conditions  tended  to  decline  as  a  general  pattern  as  lesson-posttest  question 
similarity  decreased. 
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Despite  its  informational  properties,  feedback  by  itself  does  not 
necessarily  increase  depth-of-processing  of  the  material  being  learned.  In 
fact,  a  possible  disadvantage  of  feedback  may  be  that  of  supplanting  the 
natural  tendency  of  questions  to  stimulate  information  processing  or 
"mathemagenic  activity"  (Rothkopf,  1966),  as  the  learner  searches  memory  or 
the  text  to  find  the  answers  to  questions.    That  is,  once  the  correct  answer  is 
identified,  the  learner  may  resort  to  memorizing  or  merely  acknowledging  it 
without  engaging  in  further  processing.     In  a  similar  vein,  Andre  (1979) 
discussed  how  the  availability  of  feedback  in  text  can  short  circuit  the 
instructional  effects  of  adjunct  questions  by  allowing  subjects  to  pe 
ek  ahead  at  the  answers  and  thus  avoid  searching  the  text  to  find  the 
m  on  their  own.     Relevant  to  these  interpretations,  feedback  effects  were 
noticeably  smaller  for  the  reworded  question  forms  than  for  the  identica 
forms     It  thus  appears  that  feedback,  especially  KCR  and  delayed,  generally  did 
not  stimulate  deeper  processing  of  the  present  material,  while  promoting  only 
a  limited  degree  of  transfer  to  questions  testing  the  same  information  as  the 
lesson  questions  but  differing  in  phrasing  or  structure. 

From  an  information  processing  perspective,  AUC  feedback  would 
appear  to  offer  potential  advantages  over  KCR  as  a  result  of  requiring 
continued  involvement  with  a  question  following  an  incorrect  response 
(Dcmpsey  &  Driscoll,  1989;  Noonan  1984).    Such  activity  can  increase  depth-of- 
processing  for  the  item  (Smith,  1988),  provided  that  the  learner  is  not  just 
guessing  randomly  (Underwood,  1963).    On  the  present  task,  AUC  tended  to  be 
effective  relative  to  both  KCR  and  delayed  feedback  on  reworded  questions,  but 
was  relatively  ineffective  on  identical  questions  (see  Table  2).     This  pattern 
suggests  that  AUC  may  have  served  to  promote  higher-order  learning  of  the 
material,  as  learners  reconsidered  the  questions  they  missed  in  light  of  their 
previous  c.ror  responses  and  the  remaining  alternatives.     Further  research  is 
needed  to  explore  this  possible  function  of  AUC  as  well  as  to  reconcile  the 
mixed  findings  regarding  AUC  effects  reported  in  previous  studies  (cf,  Angell, 
1949;  Clariana,  1990;  Dcmpsey  &  Driscoll,  1989;  More,  1969). 

That  feedback  effects  tended  to  be  stronger  in  the  no-text  than  in  the 
text  condition  seems  predictable,  given  that  the  former  subjects  were 
completely  dependent  on  the  adjunct  questions  to  learn  the  information.  This 
result  should  not  be  interpreted  to  imply  that  learning  is  as  good  or  better 
from  questions  only  without  accompanying  text.     Although  there  is  little  doubt 
that  tests  can  teach  (e.g.,  Fisher,  Williams,  &  Roth,  1981;  Meyer,  1965;  Pressey, 
1926,  1950),  what  is  learned  will  be  restricted  by  the  particular  focus  of  the 
items  that  happen  to  be  included.    The  implication  of  this  idea,  along  with  our 
earlier  discussion  of  feedback  effects,  is  that  feedback  studies  that  employ 
identical  lesson  and  posttest  questions  narrow  the  content  domain  to  those 
specific  questions,  and,  in  the  process,  maximize  the  importance  of  the 
questions  (and  accompanying  feedback)  while  minimizing  the  value  of 
contextual  support  (e.g.,  text). 

Another  aspect  of  the  present  text  versus  no-text  comparison  was  th<* 
failure  to  support  the  predicted  tendency  (Hypothesis  3)  for  text  to  become 
more  facilitate  as  question  level  increased.     In  fact,  the  significant  support  x 
question  interaction  obtained  on  both  tests  reflected  the  opposite  pattern;  for 
example,  the  largest  difference  favoring  text  over  no-text  occurred  on  the 
lowest  level  question  type,  verbatim-identical.     The  suggestion  is  that  subjects' 

1G5 


m  185 


Feedback  Strategies 

13 


processing  of  the  text  might  have  been  at  a  fairly  low  level    An  important 
factor  in  this  regard  appears  to  be  the  difficulty  of  the  material  and  high 
reading  level  of  the  passages. 

In  summary,  the  major  findings  evidenced  in  the  present  research 
were  that:    (a)  feedback  was  generally  effective  for  learning,  but  more  so  on 
the  lower-level   (identical)  questions  than  on  the  higher-level  (reworded) 
ones;  (b)  feedback  information  had  greater  impact  in  the  absence  of 
supporting  text  than  with  supporting  text;  (c)  relative  to  other  treatments,  AUC 
feedback  tended  to  increase  in  effectiveness  and  delayed  feedback  to  decrease 
in  effectiveness  as  question  level  was  varied  from  identical  types  to  reworded 
types, 

Past  feedback  studies,  including  the  present  investigation,  have  focused 
primarily   on   comparing   learner   achievement   under   different  feedback 
strategies.     Follow-up  research  thai  gives  greater  focus  to  intervening 
learning  behaviors  (e.g.,  degree  of  task  engagement,  referencing  of  text, 
note-taking)  would  shed  light  on  the  question  of  how  information  processing 
and  study  activity  are  influenced  by  those  strategies.     The  present  completion 
time  results,  for  example,  are  suggestive  of  varied  degrees  of  task  engagement 
that  occurred  in  the  three  feedback  conditions.     Acquiring  better 
understanding  of  such  processes  should  help  to  identify  ways  of  using 
feedback  more  effectively  to  increase  the  range  and  degree  of  learning  from 
embedded  lesson  questions. 
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Tabic  1 

Text  Portion  and  Corresponding  Questioning  Levels 


Feedback 


Text  portion 

In  conductors,  the  electrons  move  easily;  in  insulators,  they  do  not,  Since  moving 
electrons  carry  energy  as  well  as  charge,  a  good  electrical  conductor  normally  is  also  a 
good  heat  conductor,  and  an  electrical  insulator  is  also  a  poor  heat  conductor 

Questioning  Levels 
Verbatim  Identical  a 

Normally,  a  good  electrical  conductor  is  also: 

&  a  good  heat  conductor 

b.  a  poor  heat  conductor. 

c.  a  good  electrical  insulator. 

d.  the  best  electrical  insulator. 

Inferential  Identical  b 

Copper  is  a  poorer  heat  conductor  than  silver,  then  copper  probably: 

a.  has  a  smaller  heat  capacity  than  silver 

b.  is  a  poorer  electrical  conductor  than  silver. 

c.  is  a  semiconductor. 

d.  is  a  better  electrical  conductor  than  silver. 

Inferential  Transformed  c 

Copper  is  a  poorer  electrical  conductor  than  silver,  then  copper  probably: 

a.  is  a  semiconductor. 

b.  has  a  smaller  heat  capacity  than  silver. 
£.  is  a  poorer  heat  conductor  than  silver, 
d.  is  a  better  heat  conductor  than  silver. 

Inferential  Paraphrased  a 

Copper  does  not  move  heat  energy  as  well  as  silver,  so  copper  probably: 

a.  is  a  semiconductor. 

b.  moves  electrical  energy  better  than  silver. 

c.  can  hold  less  heat  energy  than  silver. 

4.  does  not  move  electrical  energy  as  well  as  silver. 

Inferential  Transformed  &  Paraphrased  a 

Copper  does  not  move  electrical  energy  as  well  as  silver,  so  copper  probably: 

a.  moves  heat  energy  better  than  silver. 

b.  is  a  semiconductor. 

£.  docs  not  move  heat  energy  as  well  as  silver. 

d.  can  hold  less  heat  energy  than  silver. 


Note:  The  letter  of  the  correct  answer  is  underscored  in  each  item. 

a  Fabricated  question  used  for  illustrative  purposes 

b  Actual  lesson  question  for  the  text  passage  shown 

c  Actual  posttest  question  for  the  text  passage  and  lesson  question  shown 
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Table  2 

Treatment  Posttest  and  Retention  Test  Means  of  the  Five  Question  Levels 


Question  Levels  and  Support 


Feedback  Verbatim  Inferential  Inferential  Inferential  Transformed        Row  Means 

and  Testing  Identical  Identical  Transformed        Paraphrased         Paraphrased  Qvsrgll 

Qa     QT"  Q      QT  Q      QT  Q      QT  Q  QT 


Immediate  Test 

KCR 

5.9 

n  4 

6.1 

6.5 

"7  4 

7.1 

4.6 

3.2 

2.7 

A  4 

4.1 

0  A 

3.4 

3.6 

A  7 

4./ 

AUC 

5.5 

5.4 

C  7 

5.5 

A  A 

4.4 

0.0 

A  0 

4.B 

0  7 

6.1 

A  n 
4.0 

A  7 

Delayed 

6.4 

6.4 

6.1 

6.7 

3.5 

3.6 

3.4 

3.5 

2.3 

2.9 

4.5 

No-feedback 

2.8 

4.8 

3.3 

2.8 

2.9 

2.5 

3.1 

3.4 

3.4 

3.3 

3.2 

No-question 

2.4 

4.9 

3.6 

3.4 

2.5 

2.7 

2.6 

32 

2.4 

2.5 

3.0 

Column  means 

4.6 

5.5 

5.2 

5.1 

3.6 

3.1 

3.3 

3.6 

3.1 

3.1. 

4.0 

KCR 

4.4 

4.8 

5.0 

5.4 

3.2 

3.4 

3.3 

3.7 

2.4 

3.0 

3.9 

AUC 

4.4 

4.4 

5.6 

4.5 

4.0 

3.2 

4.1 

3.5 

3.1 

3.2 

4.0 

Delayed 

5.2 

5.0 

4.9 

5.2 

3.4 

3.3 

3.3 

2.8 

2.7 

2.3 

3.8 

No-leedback 

2.8 

4.1 

3.1 

3.2 

2.9 

2.8 

2.9 

3.0 

3.2 

3.2 

3.1 

No-question 

2.6 

4.4 

2.9 

3.2 

2.4 

2.9 

2.7 

2.8 

2.8 

2.9 

3.0 

Column  Means 

3.9 

4.5 

4.3 

4.3 

3.2 

3.1 

3.3 

3.2 

2.8 

2.9 

3.6 

Note,  Scores  could  range  from  0-8  in  each  cell. 
aQ  ■  Questions  only;  bQT  =  Text  and  Questions 
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Table  3 

Results  of  Feedback  x  Support  x  Question  Level  ANOVAs  by  Testing 


Testing 


Immediate 

Retention 

Source  of  Variance 

df 

MS 

F 

MS 

F 

Between  Subiects 

Feedback  (F) 

4 

UO  ,Uj 

i  f.  n<** 

1  O.U  J 

ZZ  .  1  0 

7.  1  J 

SunDort  (S\ 

1 

1  QO 

i.yz 

M  J 

1  QO 

7/1 

F  x  S 

J.     A  J 

4 

7.57 

1.78 

4.79 

1.97 

Error 

90 

4.24 

2.43 

Within  Subjects 
Question  Level  (Q) 

4 

101.27 

68.78** 

43.08 

65.27** 

Fx  Q 

16 

10.97 

7.45** 

5.00 

7.58** 

SxQ 

4 

6.43 

4.37* 

2.18 

3.31 

FxS  xQ 

16 

2.65 

1.80 

.90 

1.36 

Error 

360 

1.47 

.66 

*  a<.oi 

*  *     n<  .001 
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Verbatim  Inferential  Inferential  Inferential  Transformed 
Identical       Identical       Transformed   Paraphrased  Paraphrase 

Question  Level 


193 

1 94 

ERIC 


7 


I-  5 
c 

.2 

Q)  4 
0) 


2  — 


■-  ■  KCR 

▼  ▼  AUC 

•  •  Delayed 

♦ —  ♦  No  feedback 
h. — -a  No-Question 


Verbatim 
Identical 


Inferential  Inferential  Inferential  Transformed 
Identical       Transformed   Parapnrased  Paraphrase 

Question  Level 
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Identifying  A  Range  of  Performance  Improvement  Solutions  -  High  Yield 
Training  to  Systems  Redesign  ■  Through  Evaluation  Research 


Introduction 

The  effectiveness  of  any  organization  is  dependent  on  its  most  important  resources: 
people.  Systems  and  techniques  abound  for  the  purpos6  of  developing  human  resources. 
These  are  designed  to  help  an  organization  get  the  best  performance  from  some  type  of 
training.  Analysis  can  show  what  kind  of  training  is  most  appropriate  for  achieving  improved 
performance  and  also  reveal  when  training  is  not  necessary  in  order  to  achieve  improved 
performance.  One  method  that  accomplishes  both  of  these  ends  is  evaluation  research  (Geroy 
and  Wright,  1988). 

Figure  1 
Evaluation  Research  Model 


1. 


2. 

3. 
4. 


6. 


8. 
9. 


Selection  of  Evaluation  Research  Purpose 

Needs  Assessment  —  Small  Scale  Testing 


Basic  Research 
Coverage  Accountability 


 Evaluation 

 Input  Assessment 


Identify  the  Technique  to  be  Used 

Front  End  Analysis   Program  Monitoring 

Formative  Evaluation   Evaluability  Assessment 


 Policy  Analysis 

 Fiscal  Accountability 

 Economic  Analysis 


 Impact  Evaluation 

Evaluation  of  Evaluation 


Develop  Research  Questions 

Establish  a  Collaborative  Utilization  Focused  Process 
A  Compose  Task  Force 

-  All  stakeholders  represented 

-  Power  exists  to  act  on  findings 

-  Time  committed 

-  Process  and  results  valued 


B.  Structure  Task  Force  Activities 

-  Focus  clarification 

-  Methods  and  measurement  selection 

-  Review  of  instruemtns  and  strategies 

-  Data  interpretation  meeting 


Determine  Research  Focus 
Evaluation  research  with  goals 

Determine  Evaluation  Research  Strategy 
Scientific  -  hypothetical  -  deductivu 

Select  Data  Collection  Mo'ihod 
Formal  instruments,  e.g.,  survey 

Implement  Research 

Analyze  and  Report  Results 


 Evaluation  research  without  goals 

 Anthropological  -  holistic  -  inductive 


 Non-formal  instruments,  e.g., 

structured  or  non-structured  interview 


Geroy  and  Wright,  1988 


Evaluation  research  is  a  pragmatic,  program  focused  research  strategy  of  analysis  for 
decision  makers.  Its  purpose  is  to  provide  data  that  can  enable  decision  making,  through 
analysis  of  pros  and  cons,  through  prioritizing,  or  through  the  application  of  decision  making 
criteria,  etc.  What  gives  evaluation  research  its  strength  is  the  extent  to  which  the  stakeholders 
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are  involved  throughout  the  research  process.  This  involvement  yields  four  primary  benefits: 
feasibility  in  terms  of  cost,  timeliness  and  manner  of  implementation;  and  uJiiily,  accuracy,  and 
proprietorship  in  terms  of  outcome. 

Beginning  with  Phase  One:  Selection  of  Evaluation  Research  Purpose  (See  Figure  1), 
the  research  evaluators  and  the  stakeholders  "bacome  collaborating  partners  in  the  search  for 
useful  information"  (Geroy  and  Wright,  1988,  p.  23).  From  then  on,  collaboration  is  structured  at 
8ach  step  of  the  process.  This  collaboration  provides  a  means  of  monitoring  the 
appropriateness  and  relevancy  cf  each  activity  and  strategy,  as  well  as  its  feasibility  regarding 
time  and  resources.  As  a  result,  the  information  gathered  is  more  likely  to  be  on  target  in 
addressing  the  needs  of  the  stakeholders.  It  is  also  more  likely  to  facilitate  decision  making  both 
through  its  accuracy  and  the  usefulness  of  the  way  the  information  is  reported. 

Furthermore,  those  who  have  helped  structure  the  process,  as  well  as  those  from  whom 
the  data  is  derived,  have  greater  ownership  of  the  results.  Their  potential  buy-in  to  the 
recommendations  stemming  from  the  research  is  increased  (proprietorship).  In  each  case  the 
data  gathered  are  used  to  either  create,  maintain  or  improve  program  policy  and  /  or 
implementation  practices. 

This  article  describes  a  needs  assessment  process  that  identifies  a  range  of 
performance  improvement  strategies  using  the  steps  of  the  Evaluation  Research  Model. 

Overview 

"he  Professional  Development  Committee  of  a  mid-sized  community  college  in 
suburban  Philadelphia  had  the  responsibility  of  identifying  professional  development  activities 
for  the  administrators,  faculty  and  support  personnel  of  the  college.  In  light  of  concern  voiced 
over  past  professional  development  offerings,  the  committee  decided  to  use  outside 
consultants  to  help  them  plan  their  professional  deve'opment  program.  After  three  lengthy 
meetings  with  the  committee,  the  consultants  identified  five  problems  imbedded  in  the 
environment  that  were  influencing  the  decision  making  process: 

*  lack  of  agreement  among  committee  members  as  to  how  to  select  professional  development 
opportunities, 

*  limited  communication  of  information  across  functional  groups, 

*  negative  perceptions  of  previous  professional  development  efforts, 

*  perceived  unresponsiveness  of  decision  makers  to  past  suggestions  related  to  professional 
development,  and 

*  recent  loss  of  incentive  due  to  reduced  power  and  autonomy  within  the  faculty  structure. 

It  became  apparent  that  the  final  professional  development  plan  for  the  1991-92 
academic  year  had  to: 

*  involve  and  empower  all  employees  (stakeholders)  equally  in  identifying  professional 
development  needs, 

*  develop  a  means  by  which  the  planning  committee  itself  would  recognize  that  certain  non- 
training  actions  were  required  in  order  to  maximize  the  worth  of  training, 

*  elicit  data  that  would  represent  needs  of  the  employees  of  the  college, 

*  elicit  valid  data  while  facilitating  a  process  with  280  participants,  and 

*  provide  decision  makers  with  systematic  process  fc  selecting  professional  development 
activities  that  will  yield  maximum  results  for  cost  and  effort. 

Using  the  evaluation  research  strategy  (Geroy  &  Wright,  1988)  as  a  guideline,  a  needs 
assessment  was  conducted  with  280  support-staff,  faculty  and  administrators.  The  Nominal 
Group  Technique  (Debeque  &  Van  Deben,  1974),  a  comprehensive  process  for  data  gathering 
within  a  group  setting,  was  taught  to  representatives  from  each  of  these  three  groups.  These 
representatives  facilitated  the  participation  of  virtually  all  employees  at  the  college  in  identifying  a 
prioritized  list  of  items  that  would  enable  these  persons  to  "do  their  job  better"  -  from  their 
perspective.  Analysis  of  the  suggested  performance  improvement  needs  utilized  the 
Performance  Technology  Model  (Gilbert,  1978)  and  revealed  that  the  solutions  fell  into  five 
categories  of  intervention:  high  yield  training,  direction  and  flow  of  information,  resources, 
performance  incentives,  and  medium  yield  training.  Results  indicated  that  improving  employee 
performance  required  altering  information,  resource  and  incentive  systems  and  that  without 
these  alterations,  most  training  was  likely  to  have  minimal  effect. 
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Methodology 

Each  stage  of  the  Evaluation  Research  Model  was  incorporated  into  some  phase  of  the 
project.  In  clarifying  the  goals  of  the  professional  development  planning  process,  the 
Professional  Development  Committee  and  the  consultants  used  group  discussion  and  the 
Nominal  Group  Technique  as  they: 

•  Selected  and  Evaluated  the  Research  Purpose:  Needs  Assessment  -  Stage  One, 

•  Identified  the  Technique  to  Be  Used:  Front  End  Analysis  -  Stage  Two,  and 

•  Developed  the  Research  Question:  "What  do  I  need  to  do  my  job  better?"  -  Stage  Three. 

Focus  groups  were  then  held  with  representatives  from  each  functional  area  in  order  to 
identify  any  perceptions  or  problems  in  the  workplace  that  might  have  an  impact  on  the  planning 
process.  At  this  point,  the  Committee  and  the  consultants: 

•  Established  a  Collaborative  Utilization  Focused  Process  -  Stage  Four,  and 

•  Determined  Research  Focus:  Evaluation  Research  Without  Goals  -  Stage  Hive. 

With  the  data  gathered  through  committee  meetings  and  the  focus  groups,  the 
consultants  then: 

•  Determined  the  Evaluation  Research  Strategy:  Anthropological-Holistic-Inductive  -  Stage  Six 

•  Selected  the  Data  Collection  Method:  Non-formal  Instrument,  Nominal  Group  Techniquia  - 
Stage  Seven 

Assessment  Technique 

The  Nominal  Group  Technique  was  used  to  gather  data  for  the  needs  assessment 
(Debeque  and  Van  Deben,  1974).   This  technique  utilizes  a  structured  group  meeting 
conducted  by  a  group  leader  or  facilitator  in  five  steps. 
STEP  ONE: 

•  Group  members  sit  around  a  table,  but  initially,  no  talking  takes  place.  Each  individual  has  a 
sheet  of  paper  with  the  "nominal  question"  on  it.  This  question  provides  the  primary  focus  of 
the  meeting.  This  question  is  carefully  constructed  prior  to  the  meeting  in  order  to  generate 
the  required  information.  Participants,  independently  and  silently,  write  down  as  many 
answers  to  the  question  as  possible. 

STEP  TWO: 

•  After  approximately  ten  minutes,  the  facilitator ,  going  round-robin,  calls  on  each  member  of 
the  group  tn  give  one  of  his  or  her  ideas.  Each  idea  is  listed  on  a  flip  chart  and  numbered 
sequentially.  The  puipose  of  this  stage  is  to  make  sure  that  each  participant  is  given  equal 
opportunity  to  share  his  or  her  ideas,  so  that  highly  verbal  individuals  are  not  dominant.  Thus 
all  discussion  and  judgement  are  postponed. 

STEP  THREE: 

•  The  facilitator  reviews  each  idea  sequentially,  encouraging  clarification  questions, 
elaborations,  support  for  an  idea,  as  well  as  rebuttals,  or  hitch-hiking  to  new  ideas.  This 
phase  is  complete  when  all  ideas  have  been  reviewed. 

STEP  FOUR: 

•  This  stage  is  optional  depending  on  the  number  and  kind  of  responses  generated  in  Step 
Two.  When  there  is  great  overlap  of  ideas,  the  group  can  categorize  ideas  by  topic.  This  can 
facilitate  the  ranking  that  follows  in  Step  Five.  It  is  necessary,  however,  for  members  of  the 
group  agree  on  the  categories  established  and  the  items  that  go  into  them. 

STEP  FIVE: 

•  Each  participant  silently  and  privately  ranks  the  ideas  by  assigning  a  numerical  value  to  the 
idea.  Depending  on  the  purpose  of  the  NGT  the  ranking  criteria  could  be  cost  of 
implementation,  feasibility,  importance,  etc.  The  highest  number  in  the  ranking  being  the 
total  number  of  the  ideas  (or  categories).  Each  member's  ranking  of  each  idea  is  recorded 
and  the  average  rank  of  each  item  is  derived.  This  yields  the  group's  priorities  in  relation  to 
the  ideas. 

The  advantages  of  the  NGT  are  numerous,  especially  when  one  of  the  goals  of  the 
activity  is  to  involve  stakeholders.  The  NGT  assures  that  each  person  has  an  equal  opportunity 
to  express  his  or  her  ideas  and  protects  against  dominant  personalities.  It  stimulates  the 
generation  of  ideas  through  silent  writing  in  Step  One  and  the  round-robin  listing  in  Step  Two, 
thus  preventing  closure  on  ideas  before  all  are  equally  considered.  All  participants  have  an 
opportunity  to  reflect  on  all  ideas  and  have  their  questions  and  concerns  addressed.  Silent 
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ranking  gives  equal  weight  to  each  opinion  during  decision  making  and  reduces  peer  pressure 
to  support  one  idea.  Furthermore,  the  NGT  is  cost  and  time  effective.  (Scott  and  Deadrick, 
1982) 

Implementation,  Stage  Eight  of  the  Evaluation  Research  Model,  took  place  in  several 
steps.  Representatives  from  each  functional  group  were  identified  as  facilitators  of  the  NGT. 
These  volunteers  were  trained  for  one-half  day  in  how  to  facilitate  the  NGT.  Finally,  all  employees 
of  the  college  were  gathered  for  a  day  and  a  half  during  which  three  rounds  of  the  NGT  were 
conducted. 

On  the  morning  that  all  employees  meet  together,  the  Needs  Assessment  Process  was 
introduced  and  ice  breaker  activities,  focusing  on  the  mission  of  the  college,  were  conducted. 
These  activities  were  designed  to  encourage  the  free  flow  of  ideas  during  the  afternoon  NGT 
sessions. 

After  lunch,  each  functional  group,  administrators,  support  staff  and  faculty,  met  in  a 
different  location.  There  they  divided  into  groups  of  ten  where  the  trained  volunteers  facilitated 
NGT  Round  One:  Identify  Needs  and  Round  Two:  Prioritize  Needs.  The  question  upon  which 
the  NGT  was  focused  was  "What  do  I  need  to  do  my  job  better?"  All  data  from  both  Rounds  were 
collected  and  volunteers  from  the  support  staff,  some  members  of  the  Professional 
Development  Committee  members  and  the  consultants  worked  until  10:00  pm  organizing, 
typing  and  copying  ALL  responses. 

The  next  morning  each  of  the  280  participants  received  a  copy  of  the  responses  of  his  or 
her  functional  group,  as  well  as  the  responses  of  the  other  two  groups.  This  served  several 
purposes.  It  facilitated  Round  Three:  Generate  Solutions,  b>  increasing  the  likelihood  of 
constructive  solutions.  For  instance,  a  review  of  all  of  the  responses  might  reveal  that  some  of 
the  needs  that  might  have  appeared  valid  at  first  glance,  were  less  so  when  compared  to  need* 
from  other  groups.  The  quick  turn  around  also  demonstrated  responsiveness  on  the  part  of  the 
committee  to  employee  suggestions. 

By  the  end  of  the  day  and  a  half,  the  project  had  evolved  through  the  first  eight  stages  of 
the  Evaluation  Research  Model  and  utilized  the  Nominal  Group  Technique  in  three  separate 
contexts:  establishing  the  goals  of  the  project  (15  participants),  training  the  facilitators  (40 
participants),  and  conducting  the  needs  assessment  (280  participants). 

Stage  Nine:  Analyzing  and  Reporting  Results  was  carried  out  over  the  six  weeks  that 
followed.  During  this  time,  The  Professional  Development  Committee  categorized  all  of  the 
needs  and  solutions  using  the  Performance  Technology  Model.  The  resulting  categories 
facilitated  the  decision  making  analysis. 

Analysis  Model 

The  premise  of  performance  technology  is  that  there  are  a  variety  of  factors  in  the 
workplace  that  have  bearing  on  performance  effectiveness  and  that  appropriately  adjusting 
these  factors  can  yield  exemplary  performance,  often  without  training.  Thus,  it  behoves  those 
responsible  for  the  performance  of  others  to  identify  environmental  and  individual  factors  in  a 
given  workplace  that  will  not  only  improve  performance,  but  also  maintain  exemplar  quality 
without  costly  training. 

These  factors  are  perhaps  best  represented  by  the  Behavior  Engineering/PROBE 
Model  developed  by  Thomas  Gilbert  (1978,  1982)  which  is  the  basis  of  the  Performance 
Technology  Model  (See  Figure  2.).  This  model  identifies  the  kinds  of  things  that  can  be  done  to 
improve  and  maintain  performance,  one  of  the  primary  goals  of  any  professional  development 
project. 

It  is  Gilbert's  claim  that  any  job  that  is  supported  in  all  six  of  the  areas  of  the  Model  should 
"carry  a  guarantee  of  high  competence,  provided  that  management  was  structured  so  as  to  really 
deliver  these  things  and  had  a  clear  focus  on  the  mission  of  the  job  in  the  first  place"  (1978,  p. 
87). 

Research  and  experience  verify  that  without  good  needs  assessment,  training  is  likely  to 
be  the  performance  improvement  intervention  of  choice.  When  training  is  used,  but  it's  not  the 
most  effective  solution;  wants  instead  of  needs  are  usually  being  addressed.  At  the  same  time, 
the  value  of  training  yields  less  than  optimal  return  on  the  dollar  and  time  investment,  and  has 
little  long  term  impact  on  the  organization  as  a  whole  (Kaufman,  1986). 

Strategies  related  to  evaluation  research  and  performance  technology  have  proven 
effective  in  both  carrying  out  "front  end"  identification  of  needs  (Geroy  &  Wright,  1988;  Gray 
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1987;  Debeque  &  Van  Deben,  1974  )  and  deciding  how  to  address  the  needs  so  that  the 
training  that  is  eventually  done  will  have  high  yield  to  the  trainee  and  the  organization  (Earle, 
1990).  Performance  technology  helps  to  indicate  where  training  is  not  the  best  choice  of 
intervention  and  if  training  is  the  best  intervention,  what  kind  of  training  is  best  cuited  to  meet  the 
need. 

Figure  2 
Behavior  Engineering  Model 


INFORMATION 

INSTRUMENTS 

INCENTIVES 

1  *  Relevant  and  frequent 
feedback  about  the 
adequacy  ot 

1.  Tools,  resources  and 
materials  designed  to 
achieve  performance 
needs. 

1 .  Adequate  financial 

incentives  made  contingen 
upon  performance. 

performance. 

Z  Nonmonetary  incentives 

2.  Descriptions  of  what  is 

available. 

expected  of  performance. 

3.  Career  development 

3.  Clear  and  relevant 

opportunities. 

guides  to  adequate 

performance. 

KNOWLEDGE 

CAPACrTY 

MOTIVES 

1 .  Systematically  designed 

1.  Selection  of  qualified 

1 .  Assessment  of  people's 

training  that  matches 

personnel. 

willingness  to  work  for 

requirements  of 

available  incentives.  I 

exemplary  performers. 

(Gilbert,  1978) 

Results 

The  solutions  generated  from  Round  Three  of  the  Nominal  Group  Technique  were 
tabulated  by  the  volunteers  over  a  six  week  period.  Lists  of  prioritized  needs  and  suggestions 
were  then  organized  by  kind  in  order  to  facilitate  analysis.  For  instance,  all  suggestions  that  had 
to  do  with  the  need  for  clarification  or  revision  of  the  budget  were  grouped  together.  This  is  also 
a  category  of  solutions.  The  process  of  grouping  was  conducted  separately  for  faculty,  support 
staff  and  administrators. 

At  the  conclusion  of  this  process,  three  separate  lists  of  needs/solutions  were  forwarded 
to  the  consultants.  Review  of  the  data  indicated  that  the  groups  of  ideas/needs  could  be 
combined  into  at  least  one  of  five  categories,  depending  on  the  type  of  solutions  that  would 
meet  the  need. 

Category  One:  High  Yield  Development  Programs 

•  The  needs  in  this  category  could  be  met  by  effectively  designed  training  that  matched  the  skill 
and  knowledge  requirements  of  a  specific  job  function.  The  term  high-yield  indicated  that  the 
benefit  from  the  program  was  greater  than  the  cost  of  implementing  the  program. 

Category  Two:  Information 

•  Three  general  solutions  met  the  needs  in  this  category.  The  first  solution  was  to  provide 
descriptions  of  what  was  expected  of  performance.  Second,  to  develop  clear  and  relevant 
guides  for  job  performance.  Third,  to  provide  relevant  and  frequent  feedback  about  the  level  of 
performance  relative  to  the  expectations. 
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Figure  3 
Findings:  Administration 


NEED  THEMES 

SOLUTION  CATEGORIES 

High-Yield 
Development 
Pvogwns 

Information 

Resources 

Incentives 

Moderate 

Yield 
Development 
Proa  rams 

TOTAL 
NEEDS 

1.  Training:  sensiUvity, 

Interpersonal  relations,  trust 

12 

10 

22 

2  Clarification  ot  gods, 
objectives,  policy 
procedures,  practices 

16 

16 

3.  Communication 

15 

15 

4.  More  help -atifcnd 
personnel 

11 

11 

5.  Equipment  needs 

6 

6 

6.  Reoogntion- 
appreciation 

6 

6 

7.  Budget 

a 

3 

6 

8.  Space  requirements 

3 

3 

9.  Delegation /attainment 

1 

1 

2 

TOTAL  BY  SOLUTION 

16 

35 

20 

6 

10 

87 

The  counts  in  the  cells  represent  the  number  of  times  a  need  was  indicated. 


Category  Three:  Resources 

•  This  category  involved  needs  that  could  be  solved  by  more  equipment,  better  tools,  a  more 
effective  working  environment,  or  other  solutions  that  require  additional  funding. 

Category  Four:  Incentives 

•  Needs  that  could  be  met  by  adequate  financial  and  non-financial  incentives  fall  into  this 
category.  Optimally,  these  incentives  should  be  based  on  performance.  Also,  career- 
development  opportunity  solutions  addressed  these  needs. 

Category  Five:  Moderate-Yield  Development  Programs 

•  This  category  was  similar  to  Category  One  except  for  the  yield  derived  from  the  implementation 
of  the  program.  The  cost  of  the  program  would  not  have  been  balanced  by  the  benefit.  This  was 
probably  due  to  another  related  need  that  fell  into  the  Second,  Third  or  Fourth  Categories.  It  was 
recommended  that  programs  in  this  category  not  be  implemented  until  the  other  related  needs 
were  met.  Once  the  related  needs  were  met,  however,  the  yield  of  the  program  might  change, 
or  the  need  for  the  program  might  be  eliminated  entirely. 

These  categories  represent  four  of  the  cells  of  the  Behavior  Engineering  Model. 
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Figure  4 
Findings:  Support 


NEED  THEMES 

SOLUTION  CATEGORIE 

s 

High-Yield 
Development 
Programs 

Information 

Resources 

Incentives 

Moderate 

Yield 
Development 
Programs 

TOTAL 
NEEDS 

1.  Equpnv**  needs, 

Environment  space  needs 

3 

30 

33 

Z  Communication: 
Cooperation  between 
departments 

4 

21 

3 

28 

3,  Unfomvtyof  pofctoand 
practices 

17 

17 

4,  ReoognMon/reepect 
appreciation 
reward 

employee  identification 

8 

4 

1 

13 

TOTAL  BY  SOLUTION 

4 

49 

30 

4 

4 

91 

The  counts  in  the  cells  represent  the  number  of  times  a  need  was  indicated. 


In  comparing  the  categories  across  groups,  it  is  interesting  that,  in  each  case,  information 
needs/solutions  were  offered  more  than  any  other,  and  that  resource  needs/solutions  were 
second.  This  is  consistent  whh  the  experience  of  many  performance  technologists  who  often 
find,  during  the  course  of  needs  assessments,  that  information,  in  the  form  of  clearly  defined 
performance  expectations,  policies,  procedures  and  feedback,  is  the  greatest  opportunity  for 
improvement  related  to  performance  improvement.  (Gilbert,  1991) 

Providing  information  is  less  time  consuming  and  less  costly  than  providing  training.  It  is 
also  more  likeiy  to  result  in  performance  improvement.  Training  often  occurs  when  an 
information  problem  still  exists.  Employees  will  still  not  be  clear  on  performance  expectations 
and  do  not  receive  feedback  that  will  help  them  assess  their  progress  towards  performance 
expectations.  In  these  situations,  they  simply  canmt  make  maximum  performance  improvement. 
This  results  in  frustrated  employees  and  a  frustrated  organization. 

As  a  resuft  of  tha  project  findings  and  recommendations,  the  Professional  Development 
Committee  undertook  I  j  following  follow-up  activities. 

•  Conduct  re-orientation  program  for  all  employees  (INFORMATION) 

•  Use  re-orientation  program  as  the  new  employee  orientation  program  (INFORMATION) 

•  Implement  training  identified  as  high  yield:  (TRAINING) 

-  Interpersonal  Problem  Solving  for  administrators 

-  Communication  Skills  for  faculty  and  support  afJ 

-  PC  Teaching  Methods  for  faculty 

-  Delegation  Skills  for  administratior 

-  Budget  Manning  for  administration 
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Figure  5 
Findings:  Facu'.iy 


NEED  THEMES 

SOLUTION  CATEGORIES 

High-Yield 
Development 
Programs 

Information 

Resources 

Incentives 

Moderate 

Yield 
Development 
Programs 

TOTAL 
NEEDS 



1.  ftoney/resou';es  budget 

64 

12 

76 

Z  Dtecipiine  kiontificafoo  as 
with  fonnor  Dept  structure 
-  specific  staff  development 
activities  and  p  ograms 

8 

25 

24 

b7 

3.  con-ftuinicaijon 

a  interdepartmental  contacts/ 
cooperation 

b.  clarifying  update  purpose 

a  op*,,, ofum  issues  to 
stimulate  communication 

d.  strerrline  current  College 

bureaucratic  procedures 

e.  budgeting  process 

18 

1 
5 

6 

9 

40 

4.  Equips* needs 

3 

3 

22 

3 

31 

o.  oeffKMienrnnflDon  issues- 
input  in  decision  making 

24 

OA 

—  1 

6.  Professional  vs.  empbyee 
identity  -  administrative 
ytacutty  dialog  &  relations 

19 

19 

7.  Training:  PCs,  teaching 

12 

2 

5 

19 

8.  HowtodeaJwthspedaJ 

3 

4 

3 

2 

12 

9.  Recognition 

4 

4 

10.  Team  building 

2 

2 

4 

TOTAL  BY  SOLUTION 

26 

118 

86 

40 

13 

286 

The  counts  in  tha  cells  represent  the  number  of  times  a  need  was  indicated. 


•  establish  a  new  committee  to  oversee  non-ir.otructional  performance  improvement  solutions 

•  Present  an  overview  of  performance  technology  to  the  Professional  Development  Committee 

•  Review  results  of  training  and  ncii-instructional  interventions  after  one  year 

The  solutions  reportpc)  in  the  resources,  incentives  and  information  categories  were  not 
immediately  addressed  by  the  committee  for  two  reasons.  Their  mission  had  been  to  plan 
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professional  development  activities  within  the  traditional  paradigm  of  professional  development. 
Once  iheir  vision  of  professional  development  had  been  expanded  to  include  non-training 
factors  that  support  performance  improvemb  ...  they  felt  that  the  most  responsible  strategy 
would  be  to  have  a  new  committee  established  that  could  focus  exclusively  on  non-instructional 
performance  improvement  solutions.  Secondly,  the  original  committee  was  not  empowered  to 
authorize  the  budget  expenditures  that  mighi  be  necessary  to  implement  resource  and 
incentive  solutions. 

Conclusions 

The  benefits  from  this  project  are  numerous.  Evaluation  research  (Geroy  &  Wright, 
1988)  facilitated  u.  decision  making  regarding  the  relative  worth  of  and  priority  of  actions  to 
promote  both  professional  growth  and  institutional  development.  Without  the  project,  the 
Professional  Development  Committee  would  have  selected  training  activities  that  might,  or  might 
not,  have  addressed  some  of  the  needs  of  the  college  employees.  Certainly,  they  would  not. 
have  considered  nor,  instructional  needs. 

All  of  the  challenges  that  were  identified  by  the  consultants  at  the  beginning  of  the 
project  were  addressed: 

•  All  employees  (stakeholders)  were  involved  equally  in  identifying  professional  development 
needs.  This  was  particularly  significant  for  the  support  staff,  whose  work  keeps  an  organization 
moving.  It  was  the  first  time  they  felt  anyone  had  listened  to  their  Concerns  or  ideas. 

•  The  planning  committee  did  recognize  that  certain  non-training  actions  were  required  in  order 
to  maximize  the  worth  of  training 

•  Data  was  elicited  from  all  280  employees  that  represented  both  wants  and  needs  of  the 
employees  of  the  college.  The  degree  of  overlap  of  needs/solutions  validated  both  the  data  and 
the  effectiveness  of  the  process. 

•  Decision  makers  were  provided  with  with  systematic  process  for  selecting  professional 
development  activities  that  will  yield  maximum  results  for  cost  and  effort. 

There  wero  also  benefits  related  to  the  practice  of  performance  technology.  This  project 
illustrated  the  number  and  kind  of  responses  generated  through  the  nominal  group  technique, 
as  well  as  the  range  of  professional  development  options  that  emerge  from  NGT  responses.  It 
reflected  the  cutting  edge  of  performance  technology  bv  virtue  of:  applying  of  performance 
technology  in  an  academic  seti.n«,  invoking  all  employees  of  the  organization,  addressing  the 
desire  of  the  organization  for  training  while  simultaneously,  presenting  alternative  performance 
improvement  solutions,  and  introducing  pjrformance  technology  as  the  decision  making  model 
for  all  HRD  functions  of  the  organization.  The  project  also  provided  techniques,  references  and 
examples  that  can  be  used  in  convincing  decision  makers  of  the  value  of  conducting  a  needs 
assessment. 
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Abstract 


Pa—tier  of  word  pr« 

student  ^^^^S^J^SSSSSL  allowed  integration 
other  disciplines.  In  the  present  J*™*"-    »         tory  of  9tudent  interests, 
into  stories  of  personalized  referents  from ,m ,  inw entory  o 
Stories  were  produced  at  three  levels  on  ^"^^  °™he  0,der  for  level 
one  per  day.  A  after  each  story  to  assess 

of  personalization  presentation.  A  quiz  ww  assf,ssed  using  a  post  project 

student  ^"^^ri^S.^^  to  »  for  a  fourth 

survey  which  included  choice  of  the  level  «i pe  s  for  the 

story.  Scores  were  analyzed  using  ANOVA.  ™n'e™™e"'    ...     th  treatments 

ind7viduanyperson^^^ 

and  statistically  significant.  Results  snow  uiat  p« > 

materials  can  increase  student  comprehension  of  materials  read. 
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Effects  of  Levels  of  Personalization 
on  Reading  Comprehension 


Reading  comprehension  has  been  a  topic  for  research  since  the  early  1900's 
when  ThSke  concluded  that  comprehension  is  thinking  and  "lies  on 
wnen  luumu  1Q7Q«.   pesearchers  have  continued  to  study 

W73  Edwards,  1981;  Marshall,  1984)  and  motivational  factors  (den  Heyer,  1981, 

S EMhth  wade  Hispanic  students  in  a  personalized  treatment  group  scored 
JfiffiE  on  the  constructed-response  post-Ust  for  one-step  and  two  step 

mb£^^ 

mean  score  gains  on  mathematical  word  problems  over  scores  on 
unmodified personalized  stories  taken  from  a  commercially  produced 
a>     f  -!!™«„  J  found  that  noor  readers  did  better  on  tests  of  personalized 

^^^^  ^0  -d  the  ^f  thPo°s°er 

riders'  scores  showed  substantial  increases  when  compared  to  the  scores  of  those 

Xtld  ^stories  in  the  standard,  unmodified  ^  No  W  ^und 
among  scores  for  children  of  average  reading  ability  on  this  variable 
3      "    personalization  works,  then  why  don't  we  use  it  more?  Probab y  the  best 
answer  to  this  question  is  that  personalized  materials  have  been  difficult  to 
Reduce  Creating  a  unique  version  of  a  story  for  each  studenUnte,hdb 
than  creating  one  story  and  duplicating  sufficient  copies  for  all  students  to  read. 
BrTcken  (X  sugg^ted  that  for  conventional  classroom  use  a  level  of 
llnoMzation  might  be  achieved  by  incorporating  the  names  of  several 
?Wmate  Tin  a  sinele  story  rather  than  creating  separate  stories  for  each  studen 
* E Te  Z:^Zr^s  research,  microcomputers  were  NStbeginnmg 
to  become  available,  and  under  typical  conditions  each  story  would  have  been 

indiVidRoss  and^and  (1987)  call  for  the  automation  of  materials  preparation 
which  iX«teVi^rii»tton.  A  portion  of  the  present  pro  ect  was  designed  to 
rccomphsh  that  goal.  As  in  other  studies  involving  personalization,  a  student 
fa  oe  anh  cal  survey  was  used  to  gather  information  relative  to  each  student  After 
entering  these  Ss  into  a  file,  a  computer  mail  merge  was  used  to  combine  the 
stot  une  vrith  referents  from  the  student  data  file.  The  strategy  developed  in  this 
tody  Xws  for  flexible  placement  of  any  or  all  items  and  *^m*»  branching 
to  alternate  passages  based  on  the  contents  of  any  item  of  the  student  date  file, 
to  f         gcores  from  the  ^ade  8  testing  with  the  Iowa  Basic  Skills 

Test  determined  p^JSnt  in  Chapter  I  reading  classes.  Additional  test  scores  in 
reading  prior  to  participation  in  the  project  (ninth  grade,  lower  quartile  in  reading 
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on  a  nationally  normed  test)  established  the  eligible  pool  of  students.  A  second 
^oup  of  students  came  from  two  Communications  Classes  (eleventh  grade two 
Sing  £ade  levels  below  class).  Thus,  the  participating  students  were  from  a 
sample  of  "poor  readers"  from  grades  9  through  11.  j:fforOT1t 
The  purpose  of  the  project  was  to  examine  the  effectiveness  of  three  different 
levels  o  Personalized  reading  materials  on  student  reading  comprehension  and  to 
^ItV^m^LeihJfor  creating  personalized  reading  materials  for  one  or 
more  students  using  existing  computers  and  software. 

more  Btudents  u  &M  gender  of  gubject  were  used  as  the  independent 

variables.  Inc1'  'dual  scores  on  thematic  tests  after  each  story  were  used  as  the 
dependent  me^'uro  of  student  comprehension. 


Method 


^^Twenty-six  students  in  the  ninth,  tenth,  and  eleventh  grades  from i  a 
suburban  high  school  in  the  southwestern  United  States  were  included  in  tins  study. 
Tt  student!  were  in  four  classes  taught  by  two  teachers.  Subjects  were  in  two 
Rpfldine  classes  (n  =  8),  and  two  Communication  classes  (n  =  18).  Both  the  eight 
feading^uS    tV and  the  18  communication  subjects  were  balanced  evenly  by 
«Ser  Ja  total  of  13  boys  and  13  girls.  The  students  n  the  Reading  lasses  and 
ft i  th  e Communication  classes  were  also  identified  as  "At  Risk"  students  due  to 
achievement  and  attendance  histories. 

■Mnt,ftriAS20-item  biographical  questionnaire  was  used  to  collect  personal 
information  from  each  student.  Included  were  questions  asking  for  names  of 
frienTs  wno  are  boys,  friends  who  are  girls,  favorite  color,  sports  participate in, 
and favorite  foods  Other  questions  asked  about  classes  students  liked ,  most  and 
kast  Question  selection  and  phrasing  were  substituted  to  maximize  the  likelihood 
that  a"  Participants  would  be  able  to  answer  each  question  easily  and  minimize 
Huation's  where  no  answer  was  possible  for  an  individual.  For  example  one 
question  asked  for  the  name  of  a  favorite  relative,  rather  than  the  name  ol a 
favorite  aunt  (or  uncle,  etc.),  since  all  students  have  relatives,  but  not  all  students 
mikht  have  an  aunt  or  other  specific  relative.  . 

Three  shorts*  ries  were  created.  The  first  person  narrative  stories  dealt 
with  dailv  school  experiences  common  to  high  school  students.  By  replacing  key 
words  wiih  appropriate  personalized  referents,  one  of  the  three  levels  of 
penalization  was  incorporated  into  each  story.  The  stories  were  modified I  as 
necessary^to  achieve  a  consistent  sixth-grade  reading  level  as  measured  with  the 
Frv  Readability  Formula  (Fry,  E.  A.,  1968). 

Three  levels  of  personalization  were  used  in  the  study;  non-personalized 
group  personalized,  and  individually  personalized.  The  three  levels  were  selected 
to  represent  three  common  levels  of  personalization  that  might  be  used  with 
materials  read  bv  students  in  a  classroom. 

The  non^ersonalized  version  of  the  story  used  nouns  and  pronouns  such  as 
teacher  car  and  my  friend.  The  stories  were  approximately  400  words  in  length 
£fd I  were 'printed  in  a  two-column  format  on  one  side  of  an  8  1/2  x  11  inch  paper. 

The  group  personalized  level  was  created  by  substituting  the  highest 
frequency  item  by  classroom  for  each  question  on  the  student  ■^y^^ 
selected  nouns  and  pronouns  in  the  story.  For  example,  the  sentence  . 
and  I  went  to  see  »  movn."  becomes  "fikky.  and  I  went  to  see  fiances  with  Wolves. 
"Ricky"  and  "Dances  witl  Wolves"  had  the  highest  response  frequency  in  the 

student  surveys.  ,  ,  . 

In  the  individually  personalized  version  the  structure  was  similar, 
however,  referents  from  the  individual  student's  inventory  were  used  for  merging 
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with  the  story  shell.  The  non-personalized  sentences,  "Sunday  wasn't,  too  exciting, 
but  I  did  get  to  talk  with  my  uncle  for  a  while  and  later  I  talked  for  about  an  hour  on 
the  phone  with  a  friend.  We  had  dinner  while  we  watched  TV."  became  "Sunday 
wasn't  too  exciting,  but  I  did  get  to  talk  with  uncle  Carl  for  a  while  and  later  I  U'lked 
for  about  an  hour  on  the  phone  with  Chrissv.  We  had  pizza  for  dinner  while  we 
watched  Ths  WnrlH'e  Fnnniftst  Hon™  Video's."  The  name  of  a  particular  student's 
favorite  relative,  a  friend  who  is  a  girl,  favorite  food,  and  favorite  TV  show  were 
incorporated  into  the  story.  In  the  highly  personalized  version,  each  student's  story 
was  unique,  using  referents  from  that  student's  own  inventory. 

The  stories  were  adjusted  so  that  each  contained  thirty  placeholders  for 
referents  of  generic,  group  personalized,  or  individually  personalized  items. 

Each  story  was  followed  by  a  quiz.  None  of  the  questions  asked  for 
information  that  would  come  from  student  inventory  items  nor  were  any  questions 
based  on  opinion.  The  questions  were  content-dependent  and  required 
comprehension  of  the  unchanging  (non-referent)  elements  of  the  story. 

Procedures  , ,  ,  .    A,  .  ,. 

Students  completed  a  20-item  biographical  inventory  during  their  reading 

class  one  week  prior  to  reading  the  first  story.  Teachers  were  informed  of  the 
necessity  to  have  all  items  filled  in  and  to  tell  the  students  that  the  information 
would  be  used  to  construct  stories  that  the  students  would  be  reading  the  following 
week. 

The  students  read  one  story  per  day  during  the  last  15  minutes  of  their 
reading  class.  In  order  to  guard  against  any  influence  for  order  of  presentation  of 
treatments,  the  three  levels  of  personalization  were  blocked  so  that  each  possibility 
of  order  of  presentation  was  distributed  randomly  and  equally  among  the  students. 
After  reading  the  story,  students  completed  the  ten-item  quiz  without  referring  back 
to  the  printed  story. 

Upon  completion  of  all  three  stories,  students  again  received  copies  of  the 
three  stories  they  had  read  and  were  asked  to  choose  the  level  of  personalization 
they  preferred  for  a  fourth  story.  The  attitudinal  survey  asked  questions  about  how 
the  students  liked  the  stories,  and  whether  they  would  like  to  read  more  stories 
H^rsonalized  at  levels  similar  to  the  ones  they  had  read.  One  question  asked  which 
story  they  liked  least.  An  open-ended  question  asked  why  the  student  selected  each 
story  identified  by  the  student  as  liked  or  disliked. 

All  quizzes  used  in  the  study  were  scored  by  a  single  checker.  The  answer 
sheets  carried  no  indication  of  level  of  personalization  for  story  presentation. 
Qiterinn  Measure 

The  student's  number  correct  out  of  10  questions  on  each  of  the  three 
constructed  response,  thematic  quizzes  was  recorded  as  a  measurement  of  student 
comprehension. 
Design  and  Tints*  Analysis 

The  experimental  design  was  a  3  (personalization)  x  2  (gender)  factorial 
repeated  measures  design. 

Results 

Mean  scores  for  personalization  of  reading  materials  were  as  follows: 
individually  personalized  8.26,  group  personalized  6.19,  and  non-personalized 
5.61.  A  one-way  repeated  measures  analysis  of  variance  was  performed  to  test  for 
differences  related  to  level  of  personalization  (personalized,  gvoup  personalized,  or 
non-personalized)  on  story  quizzes.  The  obtained  E  ratio  was  statistically 
significant,  E(2,  77)  =  2.52,  c  <  0.001,  with  an  eta2  value  of  .26.  A  Tukey  HSD  test 
revealed  that  the  scores  were  significantly  higher  when  the  story  was  individually 
personalized  than  when  it  was  either  group  personalized  or  non-  )ersonalized  ,  but 
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that  the  means  for  group  personalized  and  non-personalized  did  not  differ 
S^^mZnZ  ?or0gender  and  personalization  crossed  with  gender  were 

n0t  8te™t^  survey  indicated  that  81%  of  the  students 

selected  an  individually  personalized  version  for  the  fourth  story  to  be  read,  15% 
Tele  ted  a  g^oup  personalized  story,  and  seven  percent  selected  a  nonjersonahzed 
stoS  The  post  project  attitudinal  survey  question  4  asked  why  tfie  student  hked  the 
personalized  story.  Twenty  eight  responses  were  given  noting  that  the  story 
included  friends  (several  subjects  provided  more  than i  one Response  in  this 
category),  15  liked  that  the  story  talked  about  them,  and  11  liked  that  the  story 
included  things  that  the  student  liked  to  do.  Question  6  asked  why  the  studen  didn  t 
like  the  non-personalized  story.  Sixteen  students  said  it  was  boring  15  said  it 
didn't  relate  to  me,  and  six  said  the  story  wasn't  true.  Results  on  post  treatment 
quizzes  show  that  students  score  higher  when  given  in  dm  dually ^personalized 
treatments  for  each  story  and  that  group  personalized  treatments  do  not 
significantly  improve  scores  over  non-personalized  treatments. 

Discussion 

Individual  personalization  clearly  had  a  positive  effect  on  student  reading 
scores.  One  teacher  in  the  study  commented  that  as  the  students  began  to  read  the 
first  story,  he  caw  a  smile  come  to  the  face  of  several  students  as  they  read.  One  of 
the  students  who  received  an  individually  personalized  version  of  the  story  spoke 
aloud  saying  that  now  he  knew  what  the  student  survey  was  all  about. 
Subsequently  a  second  student  who  received  a  non-personalized  version, 
commented  that  his  story  didn't  have  the  names  of  his  friends  in  it  and .  wanted £ 
know  why.  At  the  conclusion  of  the  study,  this  class  asked  the  teacher  if  they  could 

keen  the  personalized  stories.  ..-«..  * 

Earlier  suggestions  that  group  personalization  might  be  effective  were  not 
supported  by  data  gathered  in  this  study.  The  system  used  to  print  all  of  the  stories 
in  the  study  required  additional  work  to  produce  the  group  personalized  version 
because  it  had  to  be  determined  separately  which  referents  had  the  highest 
frequency  for  each  class.  All  stories  were  printed  directly  from  the  computer.  A 
further  consideration  against  group  personalization  was  that  a  few  students 
objected  to  being  placed  into  groups,  or  having  certain  likes  that  were  not  their  own 
used  in  the  story.  For  example,  one  referent  was  for  type  of  music  liked.  Rap  music 
had  the  highest  referent  frequency  in  one  class,  and  therefore  was  the  inserted 
referent  in  the  group  personalized  story  for  that  class.  Some  students  objected 
stating  strongly  that  they  did  not  like  it  when  the  story  made  it  look  as  though  they 
liked  rap  music.  Other  elements  of  the  story  created  artificial  friendships  which 
some  students  objected  to,  usually  when  cultural  or  social  barriers  were  crossed. 

In  a  1986  article  on  adapting  material  to  student  interests,  Ross, 
McCormick  and  Krisak  (1986)  state,  "A  final  issue  concerns  the  practicality  of  the 
adaptive  strategy  used.  Realistically,  few  teachers  would  have  the  time  or 
inclination  to  prepare  alternative  sets  of  materials  to  represent  different  contexts. 
One  possible  reason  for  this  type  of  concern  involved  the  lack  of  readily  available 
hardware  and  software  to  meet  the  challenge  of  systematically  producing 
individually  adapted  materials.  The  system  developed  for  this  project  made  use  ot 
hardware  and  software  commonly  available  in  schools,  but  requires  only  a  single 
computer  end  printer.  The  printed  stories  were  thus  able  to  be  used  in  normal 
classr.oms  without  need  to  schedule  computer  lab  time.  For  instructors  who  are  not 
yet  comfortable  enough  with  computers  and  are  reluctant  to  take  their  classes  into  a 
lab  setting,  this  system  can  be  used  outside  of  the  classroom  and  the  materials 
brought  to  the  classroom.  Materials  created  with  this  system  can  also  be  used  as 
homework  assignments  and  do  not  require  the  student  to  have  access  to  a  computer. 
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The  benefits  of  individual  personalization  of  student  materials  can  thus  be 
achieved  with  minimal  investment  in  compute;  equipment. 

The  referents  used  in  the  stories  were  supplied  by  the  students  themselves. 
Inclusion  of  these  items  into  a  story  shell  that  was  structured  around  a  familiar 
setting  the  student's  own  classroom,  created  a  more  easily  assimilated  reading 
environment.  Steffensen,  Joag-dev,  and  Anderson  (1979)  concluded  that  the 
schemata  embodying  background  knowledge  about  the  content  of  a  discourse  exert 
a  profound  influence  on  how  well  the  discourse  will  be  comprehended,  learned,  and 
remembered".  Since  the  students  supplied  parts  of  the  story,  they  had  a  personal 
investment  in  the  product  and  were  motivated  to  pay  more  attention  to  the  material. 

Further  investigation  of  the  effects  of  this  type  of  treatment  is  needed  to 
determine  if  the  positive  increases  evidenced  in  this  study  can  be  sustained  over 
additional  treatments.  The  transfer  of  any  gains  in  reading  comprehension  from 
this  program  could  also  be  investigated  to  see  if  comprehension  increased  for  other 
materials  read,  or  if  the  uniqueness  of  the  personalized  materials  had  a  limited 
sphere  of  motivation. 

The  strategy  used  in  this  study  is  capable  of  being  expanded  to  a  fully 
automated  system  for  producing  on-demand,  highly  personalized  printed  reading 
materials.  With  such  a  system,  students  could  receive  materials  as  they  were 
needed  rather  than  transport  a  whole  semesters  worth  of  materials  around  as  a 
textbook  This  could  be  structured  in  a  fashion  similar  to  the  industrial  practice 
known  as  JIT  -  Just  In  Time  inventory  control.  The  strategy  could  also  be  adopted 
to  producing  text  files  that  could  be  incorporated  into  a  hypermedia  based  system 
capable  of  presenting  on  screen  reading  materials  in  a  variety  of  settings. 
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A  considerable  amount  of  research  has  dealt  with  the  identification  of 
individual  differences  among  learners  and  how  best  to  approach  these  differences 
in  developing  learning  strategies  and  supporting  materials  has  led  to  active 
experimentation  and  inquiry  concerning  the  role  of  cognitive-style  variables.  The 
usefulness  of  cognitive  style  research  depends  on  its  potential  to  identify  specific 
information  processing  differences  between  several  types  of  students.  Field 
dependence-independence  has  been  examined  more  closely  than  ether  cognitive 
styles,  no  doubt  because  it  describes  characteristics  which  are  so  directly  applicable 
to  the  learning  process.  Measures  of  field  dependence  such  as  the  GEFT  employed 
in  this  study  are  arguably  tests  of  ability.  In  fact,  the  aspect  of  field  dependence- 
independence  that  is  tapped  by  the  GEFT  has  been  identified  as  cognitive 
restructuring  ability.  There  is  certainly  a  strong  visual-spatial  element  in  field 
dependence-independence  (Witkin,  1977).  For  this  reason  researchers  have 
attempted  to  design  visual  instruction  according  to  the  characteristics  of  field- 
dependent  and  independent  learners,  hoping  to  capitalize  on  strengths  and 
compensate  for  weaknesses. 

Witkin,  et.  al.  (1977)  and  his  associates  have  noted  that  certain 
individuals  interact  to  superfluous  cues  in  a  visualized  instructional  environment 
while  others  are  able  to  identify  precisely  the  critical  information  contained  in  a 
complex  visualized  environment.  These  two  orientations  called  field  dependence 
(FD)  and  field  independence  (FI)  represent  two  ends  of  a  continuum.  Field 
dependent  individuals,  when  presented  a  visualized  presentation  tend  not  to 
modify  the  structure  but  accept  and  interact  with  it  as  it  is  presented.  They  tend  to 
fuse  all  segments  within  the  visual  field  and  do  not  view  or  interact  with  the  visual 
components  discretely.  Field  independents  tend  to  act  upon  a  visual  stimulus, 
analyzing  it  when  it  is  organized  and  providing  their  own  structure  when  it  lacks 
organization. 

The  FD/FI  cognitive  style  appears  to  be  especially  important  in  the  design  of 
visually  related  information.  Although  many  studies  have  examined  the  effects  of 
visual  attributes  on  learning  (Dwyer,  1978, 1987)  few  have  studied  the  effects  of 
varied  visual  attributes  on  specific  cognitive  learning  styles.  Research  has  shown 
that  color  coding  helps  learners  organize  or  categorize  information  into  useful 
patterns  which  enables  the  learners  to  interpret  and  adjust  to  their  environment. 
Color  coding  may  be  considered  a  strategy  which  students  enhance  or  sharpen 
essential  message  characteristics  by  providing  structures  for  the  storage  of  new 
information  (Dwyer,  1978).  Dwyer  (1978),  in  an  extensive  rpyiew  of  research  on  the 
impact  of  color  vs.  black  and  white  comparison,  found  ucolor  versions  were  to  be 
significantly  more  effective  than  the  black  and  white  versions  in  facilitating 
student  achievement  of  specific  educational  objectives.  These  results  seem  to 
provide  substantial  evidence  that  colors,  in  fact,  are  a  viable  instructional 
variable"  (p.  149).  However,  a  qualification  is  noted  in  that  color  has  to  be  used 
judiciously  and  the  visuals  need  to  be  appropriate  for  the  type  of  educational 
objective  to  be  achieved. 

It  was  hypothesized  that  color  coded  visuals  (Color)  would  be  more  effective 
than  black  and  white  coded  visuals  (B&W)  in  enhancing  the  salient  visual  cues 
thereby  making  them  more  identifiable  and  instructional  to  field  dependent 
learners.  The  color  coding  would  attempt  to  compensate  for  the  restructuring  skills 
absent  in  field  dependent  learners  and  subsequently  lead  to  deeper  information 
processing  and  increased  achievement.  This  hypothesis  seems  plausible  since 
field  dependent  learners  tend  to  be  global  in  perception  and  would  be  most  inclined 
to  take  advantage  of  the  increased  structure  provided  by  the  color  coding. 

However,  do  color  enhanced  diagrams  also  have  any  effect  on  retention  on 
verbal  tests?  Previous  studies  have  investigated  the  use  of  B&W  and  Color  coded 
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illustrations  to  support  use  of  color  enhancement  over  black  and  white  illustrations 
in  answering  questions  of  a  verbal  nature,  i.e.,  terminology  and  comprehension 
(Dwyer,  1968,  1978,  1987).  However,  there  is  no  research  available  which 
investigates  the  role  of  color  on  verbal  style  questions  across  the  cognitive  style 
known  as  field  dependence  (FD),  field  independence  (FI).   Additionally,  it  is  not 
known  whether  coding  (B&W  and  Color)  and  field  dependence  interact  or  whether 
color  coding  positively  affects  the  deficiency  of  field  dependent  learners  to 
organize  and  structure  instructional  content. 

Specifically,  the  purpose  of  this  study  was  to  examine  the  effect  that  coding 
(B&W  and    )lor)  has  on  the  achievement  of  students  categorized  as  FD/FI 
leamers  a     to  determine  if  there  was  any  interaction  between  these  variables 
(field  dependency  an<i  color)  across  both  visual  and  verbal  oriented  tests 
measuring  different  educational  objectives,  namely  to  learn  terminology  and 
basic  facts,  to  identify  locations  and/or  positions,  construct  and/or  understand 
relationships  and  to  engage  in  problem  solving  activities.  It  was  anticipated  that 
the  findings  of  the  present  study  would  provide  teachers  and  designers  of 
instructional  software  with  techniques  and  aids  to  be  used  with  students  possessing 
different  cognitive  styles. 
Procedures 

One  hundred  nineteen  students  enrolled  in  a  basic  educational  psychology 
course  at  The  Pennsylvania  State  University  participated  in  this  study.  Student* 
were  classified  as  field  dependent,  field  neutral,  or  field  independent  as  a  result  of 
their  performance  on  the  Group  Embedded  Figures  Test  (GEFT),  (Witkin  1971). 
Students  were  divided  into  the  different  levels  based  on  their  mean  achievement 
level  on  the  GEFT.  The  grand  mean  achievement  by  students  on  the  GEFT  was 
13.08  with  a  standard  deviation  of  4.01.  Students  with  one  half  standard  deviation 
above  the  mean  were  considered  to  be  field  independent  (n  =  43,  x=17.0,  SD  =  .81); 
students  located  one  half  a  standard  deviation  below  were  classified  as  field 
dependent  (n  =  29,  x=  7.33,  SD  -  2.11).  Students  in  the  middle  were  classified  as 
neutral  FN  (n  =  45,  x=13.17,SD=:1.49)  (Moore  &  Dwyer,  1991).  Students  in  the  three 
GEFT  levels  were  randomly  assigned  to  two  treatment  groups.  The  subject  content 
for  the  study  consisted  of  a  2,000  word  instructional  booklet  on  the  anatomy  and 
functions  of  the  human  heart.  Each  booklet  contains  19  illustrations  which  were 
designed  to  illustrate  the  content  being  presented  verbally.  The  illustrations  in 
Treatment  I,  the  black  and  white  version,  contained  black  and  white  coded  line 
drawings  which  highlighted  the  information  and  processes  being  presented. 
Students  in  Treatment  II  received  the  same  visuals  as  did  students  in  Treatment  I; 
however,  several  different  colors  were  used  to  highlight  the  information  and 
processes  being  discussed.  The  major  independent  variables  in  the  study  were  the 
effect  that  B&W  and  Color  coding  of  information  had  on  the  information 
processing  strategies  of  students  identified  as  possessing  different  levels  of  field 
dependence. 
Dependent  Measures 

Each  student,  after  interacting  with  their  respective  instructional 
treatments,  received  two  visually  oriented  criterion  tests.  The  drawing  test 
required  students  to  draw  a  representative  diagram  of  the  heart  and  place  the 
number  of  listed  parts  (n=20)  in  their  respective  positions,  The  emphasis  for  this 
test  was  on  the  correct  positioning  of  the  verbal  symbols  with  respect  to  one  another 
and  in  respect  to  their  concrete  referents.  The  identification,  a  20-item  multiple 
choice  test,  required  students  to  identify  the  numbered  parts  on  a  detailed  drawing 
of  a  heart.  The  objective  of  this  test  was  to  measure  the  ability  of  the  student  to  use 
visual  cues  to  discriminate  one  structure  of  the  heart  from  another  and  to  associate 
specific  parts  of  the  heart  with  their  proper  names.  K-R  21  reliability  coefficients 
from  the  drawing  and  identification  tests  were  .67  and  .80  respectively. 
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Each  student,  after  interacting  with  their  respective  instructional 
treatment!:,  also  received  two  verbally  oriented  criterion  tests.  The  terminology 
test  (K-R  21,  ,.,3)  wa?  a  twenty-item  multiple  choice  test  which  evaluated  students1 
knowledge  of  facts  and  terminology  and  the  comprehension  test  (K-R  21,  .G8) 
measured  un,1  ^standing  of  the  internal  processes  occurring  within  the  heart 
during  the  diastolic  and  systolic  phases  (Moore  &  Dwyer,  1991). 
Besulta 

A  series  of  two-way  analyses  of  variance  were  used  to  analyze  the  data. 
The  independent  variables  of  color  enhancement  and  cognitive  style  were  tested 
across  each  of  the  four  dependent  measures  (drawing,  identification  terminology 
and  comprehension  tests).  An  analysis  of  variance  conducted  on  the  drawing  test 
indicated  that  there  was  a  significant  difference  in  mean  achievement  for 
cognitive  style  (FD/FI),  F  (2,  111)  =  3.99,  p  <  .05. 

Field  independent  students  (x  =  16.42)  scored  significantly  higher  on  the 
drawing  test  than  did  field  dependent  students  (x=  12.93)  on  both  the  B&W  and 
Color  coded  treatments.  Across  all  levels  of  cognitive  style  (field  dependent,  field 
neutral,  field  independent),  students  who  received  the  color  coded  illustrations  (x= 
17.35)  achieved  significantly  higher  scores  on  the  drawing  tests  than  did  students 
who  received  the  B&W  coded  illustrations  (x=  12.24),  See  Table  1  for  mean  scores. 

On  the  identification  test,  field  independent  students  (x  =  17.23)  achieved 
significantly  higher  than  did  field  dependent  students  (x=  14.41),  F  (2,111)  =  4.51,p 
<  .05.  No  significant  differences  in  achievement  were  found  to  exis<  on  the 
identification  tes<  between  students  receiving  the  B&W  and  Color  coded 
instructional  treatments  F  (1,  111)  =  3.83,  p.  >  .05.  Although  across  all  levels  of 
cognitive  style,  the  mean  achievement  scores  of  students  on  the  identification  who 
had  received  the  Color  coded  treatment  (x  =  16.76)  were  significantly  higher  than 
those  who  received  the  (B&W)  coded  treatment  (x  =  15.09).  See  Table  2  for  mean 
scores.  When  student  scores  across  the  drawing  and  identification  tests  were 
combined  into  a  total  test  both  cognitive  style  mean  score  (FD/FI),  F  (2,  111),  =  5.00, 
p.  <  .05,  and  Color  coding,  F  (2,  111)  =  24.48,  p.  <  .05  were  significant. 

On  the  terminology  test  field  independent  students  (x  =  14.54)  scored 
significantly  higher  in  achievement  than  field  dependent  students  (x=  11.24),  F  (2, 
111)  a  5.37,  p.  <  .05.  In  examining  the  differences  across  cognitive  style  levels,  no 
significant  differences  in  achievement  were  found  on  the  terminology  test  between 
students  receiving  the  B&W  and  Color  coded  treatments,  F  (1,  111)  =  .65,  p.  >  .05 
(Moore  &  Dwyer,  1991).  An  insignificant  interaction  was  found  to  exist  between 
cognitive  style  and  color  coding  (B&W  and  Color),  F  (2,  111)  =  .29,  p.  <  .05.  See 
Table  3  for  mean  scores. 

As  noted  earlier,  analysis  of  student  achievement  on  the  comprehension 
test  indicated  that  field  independent  students  (x)  =  12.81)  scored  significantly 
higher  than  field  dependent  students  (x=  10.66),  F  (1,  111)  =  3.81,  p.  <  .05.  However, 
no  significant  differences  in  achievement  were  found  to  exist  on  the 
comprehension  test  between  students  receiving  the  B&W  and  Color  coded 
treatments  (Moore  &  Dwyer,  1991).  See  Table  4  for  mean  scores. 

On  the  total  criterion  test  in  which  all  four  dependent  measures  were 
combined,  a  significant  difference  was  found  to  exist  among  the  three  field 
dependence  levels,  F  (2,  111)  =  6.29,  p.  <  .05,  but  significant  differences  were  found 
to  exist  between  the  B&W  and  Color  coded  treatments,  F  (1,  111)  =  10.62,  p.  <  .05.  No 
significant  interaction  was  found  to  exist  between  treatment  type  and  the  three  field 
dependence  levels,  F  (2,  111)  =  2.06,  p.  >  .05. 

Two  secondary  analyses  were  conducted  to  examine  achievement 
differences  occurring  among  the  three  field  dependence  levels  when  students 
received  only  the  B&W  or  Color  coded  treatments.  The  one  way  ANOVA  indicated 
that  significant  differences  existed  in  favor  of  the  field  independence  students  on 
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all  tests;  the  drawing  test,  F  (2,  58)  =  3.78,  p.  <  .05,  the  identification  test,  F  (2,  58)  = 
3.83,  p.  >  .05,  the  terminology,  F  (2,  56)  =  4.19,  p.  <  .05,  and  comprehension  tests,  F 
(2,  56)  =  6.81,  p.  <  .05,  when  they  received  the  B&W  coded  instructional  treatments; 
however,  when  making  the  similar  comparison  among  students  receiving  only  the 
Color  coded  treatments  no  significant  differences  were  found  to  exist  among  FD/FI 
students  on  the  four  independent  criterion  measures  [drawing  test,  F  (2,  57)  =  .60,  p. 

>  .05;  identification,  F  (2,  57)  =  1.07,  p.  >  .05,  the  terminology  test,  F  (2,  57)  =  1.60,  p. 

>  .05,  and  the  comprehension  test,  F  (2,  57)  ■  .98,  p.  >  .05]. 
Discussion 

The  results  of  the  experimental  study  support  that  the  contention  that  field 
independent  and  field  dependent  learners  differ  in  the  cognitive  processes  they  use 
as  well  as  in  the  effectiveness  of  these  cognitive  processes  as  measured  on  test 
measuring  different  education  objectives.  In  the  present  study,  field-independent 
students  s  ored  significantly  higher  than  did  field  dependent  students  on  the 
drawing,  identificaiion,  terminology  and  comprehension  tests  (Table  1-4).  These 
resuHs  might  have  been  expected  since  prior  research  (Moore  &  Bedient,  1986)  have 
found  that  field  independent  learners  tend  to  score  higher  on  criterion  measures 
which  require  the  acquisition  of  information  from  visualized  instruction  and  are 
used  to  assessing  visually  complemented  instruction.  This  finding  is  also 
consistent  with  the  previous  reviews  of  the  literature  that  have  concluded  that  field 
independent  learners  exhibit  an  active,  hypothesis-testing  strategy  towards 
learning,  whereas  field  dependent  learners  tend  to  employ  a  more  tentative  or 
spectator  approach  to  learning  (Goodenough,  1976).  Annis  (1979),  in  a  study 
investigating  cognitive  style  on  study  technique,  found  that  field  dependent 
students  did  not  score  as  well  as  field  independent  students  in  completing  items  of 
high  structural  importance  even  when  the  passage  was  well  organized.  The 
implication  being  that  field  dependent  students,  in  addition  to  receiving  a  well 
organized  passage,  may  need  explicit  structural  suppo.t  to  insure  that  they  identify 
and  interact  with  the  cri  ,cal  aspects  of  the  information  being  presented. 

In  the  present  study,  significant  differences  in  achievement  were  found  to 
exist  between  FD/FI  learners  on  the  drawing  test.  Apparently,  tl.e  use  of  color 
coding  of  the  visualization  did  not  provide  sufficient  structuring  of  the  critical 
information  to  alter  the  information  processing  level  of  field  dependent  learners 
on  this  learning  task  which  required  students  to  construct  and/or  position  the  parts 
of  the  heart  in  their  appropriate  context  on  the  drawing.  However,  on  the 
identification  test  no  significant  differences  in  achievement  were  found  between 
FD/FI  learners  who  had  received  the  Color  coded  treatments.  For  the  type  of 
learning  measured  by  the  identification  test,  Color  coding  was  instrumental  in 
providing  the  level  of  learning  structure  which  enabled  the  FD  learners  to  achieve 
at  a  level  similar  to  that  achieved  by  the  FI  learners.  Prior  research  supports  the 
notion  that  field  dependent  learners  benefit  from  the  restructuring  aspect  of 
instructional  strategies  (i.e.,  B&W  coding  to  Color  coding),  while  Fl  learners  are 
not  impeded  by  imposed  structures  (Grieve  &  Davis,  1971;  Slatterly  &Telfer,1979). 

As  noted  previously,  students  identified  as  field  independent  in  this  study 
also  achieved  significantly  higher  scores  on  the  two  verbally  oriented  criterion 
tests  than  did  students  identified  as  field  dependents.  It  appears  the  field 
independent  learners  were  able  to  identify  the  essential  salient  cues  and  impose  a 
more  meaningful  structure  on  the  perceptual  field  in  relation  to  the  content  being 
presented.  Field  dependent  learners  interacted  with  identical  visualized  content; 
however,  they  only  interacted  with  the  visualized  instruction  (perceptual  field)  at  a 
superficial  level.  An  inspection  of  the  means  achieved  on  the  terminology  (Table 
3)  and  comprehension  tests  (Table  4)  by  students  in  the  different  FD/FI  levels 
indicated  that  just  over  50  percent  of  the  total  content  to  be  measured  by  each 
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criterion  test  was  acquired  as  a  result  of  the  visual  coding  (B&W  and  Color)  of  the 
instructional  content  (Moore  &  Dwyer,  1991). 

In  analyzing  the  performance  of  students  who  received  only  B&W  coded 
treatments,  it  was  found  that  the  field  independent  students  achieved  significantly 
greater  scores  than  did  the  field  dependent  students  on  the  four  criterion  tests: 
drawing,  identification,  terminology  and  comprehension.  The  single  color 
coding  strategy  (B&W)  was  not  suP?iently  strong  enoue*  to  alter  the  information 
pn  jessing  strategies  of  the  field  dependence  learners.  However,  it  was  expected 
that  the  Color  coded  visuals  would  make  the  relevant  cues  more  obvious  to  the  field 
dependent  learners,  thereby  improving  their  achievement  on  the  verbal  tnsts.  The 
insignificant  results  on  all  four  criterion  tests  did  support  this  contention. 
Apparently,  tue  Color  coded  illustrations  provided  a  sufficient  structure  ior  the  field 
dependent  learners  to  interact  wiih  and  internalize  at  a  level  similar  to  that 
achieved  by  the  field  independents.  Field  independent  learners  who  interarted 
with  the  color  Coded  illustrations  may  have  feK  that  these  illustrations  adequately 
conveyed  the  intended  information  and  did  not  utilize  their  special  skills  to  process 
the  information  deeply.  For  example,  the  directions  to  the  ins  .ruction al  unit 
indicated  that  the  unit  was  designed  to  help  them  understand  the  functions  of  the 
human  heart.  Students,  as  they  preceded  through  both  instructional  treatments, 
may  have  felt  that  they  understood  the  content  adequately.  Performance  on  the 
criterion  tests  between  the  two  groups  might  havo  changed  significantly  if  the 
directions  in  the  instructional  book'H  had  indicated  that  they  would  not  only  have 
to  understand  the  functions  of  the  numan  heart  but  would  also  have  * °  perform  on  a 
test  measuring  knowledge  of  terminology  and  the  functions  of  the  various  parts  of 
the  heart  during  the  diastolic  and  systolic  phases.  Possibly,  specific  directions 
relating  to  precise  performance  expectations  would  have  stimulated  the  field 
independent  learners  to  use  their  special  skills  to  initiate  deeper  information 
processing  which,  in  turn,  would  have  led  to  increased  information  acquisition. 

The  results  of  this  study  indicate  that  the  concept  of  field  dependence/field 
independence  is  an  important  instructional  variable  in  the  teaching-learning 
process.  Color  coding  was  also  found  to  be  an  effective  instructional  variable  for 
maximizing  the  information  processing  acquisition  level  for  field  dependent 
learners  on  the  types  of  criterion  measures  employing  visually  oriented  tests  used 
in  this  study.  The  significant  interaction  on  the  comprehension  test  between 
treatment  and  cognitive  style  (FD/FI)  found  in  this  study  appears  to  be  a  spurious 
finding  and  needs  to  be  replicated  before  credibility  can  be  attached  to  its  existence. 
However,  on  verbally  oriented  tests,  color  coding  was  not  found  to  be  an  effective 
instructional  variable  for  maximizing  the  information  processing  acquisition 
levels  across  all  levels  of  field  dependence. 

Note:    This  paper  is  based  upon  a  paper  presented  at  the  meeting  of  the 
International  Literacy  Association,  Washington,  DC,  October,  1991. 


199 

221 


Table  1  -  Mean  Scores  -  Drawing  Test 


Field  Dependence 


Color 


va**&  white 


Total 


FI 


Neutral 


FD 


Mean 

"7  8 

14.50 

16.42 

SD 

2.57 

3.45 

3.36 

n 

25 

18 

43 

Mean 

16.96 

11.68 

14.38 

SD 

2.76 

3.86 

4.24 

n 

23 

22 

45 

Mean 

17.10 

10.74 

12.93 

SD 

3.28 

5.46 

5.66 

n 

10 

19 

29 

Mean 

17.35 

12.24 

14.77 

SD 

2.75 

4.54 

4.54 

n 

58 

59 

117 

Total 


Table  2  -  Mean  Scores  •  Identification  Test 


Field  Dependence 


Color 

_Qkn  Kfark&White 


Total 


FI 


Neutral 


FD 


Total 


Mean 

17.32 

17.11 

17.23 

SD 

2.87 

2.22 

2.59 

n 

25 

18 

43 

Mean 

16.70 

14.50 

15.62 

SD 

3.61 

3.64 

3.75 

n 

23 

22 

45 

Mean 

15.50 

13.84 

14.41 

SD 

3.75 

4.99 

4.60 

n 

10 

19 

29 

Mean 

16.56 

15.09 

15.92 

SD 

3.34 

3.98 

3.76 

n 

58 

59 

117 
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Table  3  -  Mean  Scores  •  Terminology  Test 


Field  Dependence 

fmK 

local 

Cckw 

Black&White 

V  1 

Mean 

14.48 

14.61 

14.54 

on 

4.33 

3.68 

4.03 

n 

25 

18 

43 

Neutral 

Mean 

13.22 

12.05 

12.64 

SD 

3.78 

4.08 

3.92 

n 

23 

22 

45 

FD 

Mean 

11.80 

10.95 

11.24 

SD 

4.42 

4.06 

4.13 

n 

10 

19 

29 

Total 

Mean 

13.52 

12.48 

12.99 

SD 

4.18 

4.16 

4.18 

n 

58 

59 

117 

Table  4  -  Mean  Scores  -  Comprehension  Test 


Field  Dependence  Color  Total 
 QiT  Rl»ok&  White  


FI 

Mean 

12.28 

13.56 

12.81 

SD 

3.78 

4.54 

4.11 

n 

25 

18 

43 

Neutral 

Mean 

12.83 

8.68 

10.80 

SD 

3.64 

3.81 

4.24 

n 

23 

22 

45 

FD 

Mean 

10.70 

10.63 

10.66 

SD 

5.29 

4.18 

4.50 

n 

10 

19 

29 

Total 

Mean 

12.22 

10.80 

11.50 

HD 

4.01 

4.56 

4.34 

n 

58 

59 

117 
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The  Use  of  Instructional  Design  Skills 
in  the  Mental  and  Written  Planning  Processes  of  Teachers 


Fingers  in  the  Dike 

Public  schools  in  particular  and  education  in  general  have  been  widely  criticized  in  recent 
years.  Although  criticism  is  not  new  in  the  field  of  education,  current  critics  are  focusing  their 
concerns  on  the  "products"  of  education-our  children— and  the  skills  they  seem  to  lack.  The 
feeling  is  that  our  nation  is  at  risk  because  of  our  students'  lack  of  preparation  for  entering  the 
work  force.  Our  "once  unchallenged  preeminence  in  commerce,  industry,  service,  and 
technological  innovation  is  being  overtaken  by  competitors  throughout  the  world"  partly  because 
of  the  "rising  tide  of  mediocrity"  in  the  schools  (National  Commission  on  Excellence  in  Education, 
1983,  p.  5).  Such  international  comparisons  highlight  deficiencies  in  our  current  educational 
system. 

Over  the  years  such  criticism  has  generated  both  beneficial  and  detrimental  changes  in 
our  approaches  to  education.  In  response,  the  educational  pendulum  has  swung  excitedly  from 
one  extreme  to  the  other,  advocating  this  program  or  that  activity  to  solve  the  educational 
dilemma.  However,  this  bandwagon  approach,  rather  than  providing  valid  solutions  to  the 
problems,  merely  administers  "band-aid"  treatments  to  the  symptoms. 

This  prevalent  quick-fix  mindset  reminds  me  of  a  favorite  story  from  one  of  my 
elementary  school  readers.  A  little  Dutch  boy  who  lived  in  the  flood-prone  lowlands  of  Holland 
discovered  a  small  leak  in  one  of  the  protective  dikes  which  held  back  the  waters  of  the  North 
Sea.  Although  only  a  dribble  was  visible,  he  knew  that,  before  he  could  summon  adult  help,  the 
water  would  have  eroded  a  large,  destructive  opening  in  the  earthen  bank.  So  he  took  the  only 
practical  action  possible  in  those  circumstances-he  stuck  his  finger  in  the  \i^<  in  the  dike  and 
stopped  the  leak— and  thus  averted  a  life-threatening  deluge  of  water  from  destroying  his  village. 
His  courageous  stop-gap  action,  though  effective,  was  merely  a  temporary  measure.  Eventually 
the  dike  had  to  be  rebuilt  and  strengthened  to  avoid  future  deterioration. 

It  is  thus  with  education  and  the  public  schools.  There  are  just  too  many  courageous, 
dedicated  teachers,  parents,  students,  and  administrators  with  their  "fingers  in  the  educational 
dike"— with  little  hope  of  developing  a  strong  and  effective  system  as  long  as  their  efforts  are 
concentrated  on  symptoms  rather  than  causes. 

What  are  the  causes  of  the  apparent  failure  of  our  children  to  meet  the  criteria  for 
acceptable  performance?  Have  we  lost  sight  of  our  goals?  Is  the  pursuit  of  excellence  too 
difficult?  Do  we  need  increased  budgets  and  salaries?  Are  our  teachers  only  mediocre?  Are  our 
curricula  outdated  or  unrealistic?  Answers  to  questions  such  as  these  will  be  found  as  we  adopt 
a  new  perspective  for  the  public  schools— a  change  from  the  traditional  myopic  "finger-in-the-dike" 
approach  which  has  pervaded  education  for  generations  to  a  focus  on  human  performance. 

is  indeed  time  to  break  out  of  the  current  mediocrity  cycle  which  continues  to  produce 
stud.ii  ^  who  lack  adequate  preparation  for  the  work  force  as  we  move  from  an  industrial  to  an 
information  and  service  age.  Educational  technology  is  the  catalyst  for  this  change.  No,  I  don't 
mean  that  we  merely  need  more  computers  in  the  classrooms.  Technology  involves  process.  It 
is  the  system  which  requires  our  attention.  Too  often  our  efforts  to  improve  education  have 
resulted  in  an  unrealistic  isolation  of  technological  tools  (e.g.,  interactive  video,  hypermedia,  and 
computers)  from  the  technological  system  or  process.  For  example,  although  a  recent  report  of 
a  National  Education  Association  committee  discusses  the  significant  ways  in  which  technological 
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innovations  will  significantly  improve  educational  opportunities  and  warns  against  a  piecemeal 
approach,  their  emphasis  is  still  on  "things"  (hardware  and  software)  rather  than  process  (NEA 
Special  Committee  Report,  1989),  Remember  educational  television  in  the  60's?  We  expended 
our  entire  resources  on  insulting  equipment  which  soon  began  to  gather  dust  because  we  neglected 
the  process  component.  Don't  get  me  wrong.  I  do  believe  that  such  tools  are  indeed  valuable 
resources,  but  only  when  used  in  an  effective  system  which  develops  human  competence,  only 
when  used  in  the  systematic  design  of  instruction.  Until  such  a  system  is  in  place,  promoting  one 
tool  or  another  is  simply  adding  more  fingers  to  the  dike. 


Technology  as  Process:  Using  ID  Skills 

Technological  changes  in  teacher  preparation  programs  have  emphasized  instructional 
design  theory  as  a  vehicle  for  improvement  (Reiser,  1988;  Shrock  &  Byrd,  1988;  Schiffman, 
1988;  Knirk,  1988;  Earle,  1985).  Although  this  is  a  valuable  framework  for  preparing  teachers, 
it  is  not  widely  used  in  teacher  education  programs  (Schiffman  and  Gansneder,  1987;  Rossett  and 
Garbosky,  1987;  Earle,  1989).  "Except  for  a  brief  spurt  of  activity  in  the  early  1970s, 
instructional  designers  have  not  played  a  major  role  in  higher  education  or  in  the  public  schools" 
(Reiser,  1988,  p.  5). 

A  recent  survey  of  graduate  programs  in  instructional  technology  (Schiffman  & 
Gansneder,  1987)  attempted  not  only  to  identify  characteristics  of  such  programs  but  also  to 
outline  their  involvement  in  teacher  education  or  the  public  schools.  Although  many  programs 
offered  media  and  computing  courses  for  teacher  education  majors,  those  specializing  in 
instructional  design  tended  to  offer  no  ID  courses  for  teachers.  The  same  survey  also  indicated 
that  ID  faculty  are  less  likely  to  participate  in  teacher  education  planning  or  to  have  formal  ties 
with  schools  than  those  in  the  media  and  computing  areas. 

The  rift  between  theory  and  practice  is  obvious  -  particularly  in  an  area  where  the  design 
of  appropriate  instruction  is  critical:  the  schools  and  the  teachers  in  those  schools.  The  challenge 
is  equally  as  obvious  -  bridging  the  gap  so  that  instructional  design  theory  is  part  of  a  teacher's 
repertoire. 

Teaching  Planning 

Planning  can  be  viewed  from  two  perspectives--as  a  blueprint  or  as  a  process  (Yinger, 
1979).  McCutcheon  (1980)  considered  the  complex,  reflective  mental  dialogue  which  is 
prerequisite  to  written  plans  as  by  far  the  "richest  form  of  teaching  planning"  (p.  7). 

The  teacher  planning  literature  has  attended  in  the  past  to  the  categories  of  planning 
(which,  in  many  ways  resemble  the  steps  in  the  ID  process),  the  time  frames  of  planning,  and  the 
products  or  processes  of  planning.  (McCutcheon,  1980;  Clark  &  Yinger,  1979;  Morine- 
Dershimer,  1978-9;  Peterson,  Marx,  &  Clarice,  1978;  Zahorik,  1975).  More  recent  research  has 
emphasized  the  practical  applications  of  ID  skills  in  the  planning  processes  of  teachers  (Reiser  & 
Mory,  1991;  Klein,  1991;  Martin,  1990;  Martin  &  Clemente,  1990). 

This  study  focusses  not  only  on  differences  and  similarities  between  the  written  product 
and  the  mental  process  of  teacher  planning,  but  also  on  the  systematic  framework  used  by  teachers 
in  this  process. 

The  UNCW  Program 

The  undergraduate  teacher  education  program  at  the  University  of  North  Carolina  at 
Wilmington  has  had  an  instructional  design  component  since  1976  (Earle,  1985).  This  component 
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involves  a  two-semester  sequence  with  the  first  course  concerned  with  instructional  design  and  the 
second  course  the  study  of  classroom  evaluation.  In  our  ID  course,  we  emphasize  the  importance 
of  a  systems  approach  to  planning  instruction  and  teach  our  students  fundamental  concepts, 
principles,  and  skills  (at  a  basic  level)  of  instructional  design.  In  the  companion  evaluation  course, 
we  continue  to  use  an  ID  model  to  provide  a  meaningful  context  for  classroom  evaluation, 
emphasizing  the  interdependence  of  instructional  decisions  and  evaluation  decisions.  We  are 
primarily  interested  in  teaching  our  students  a  practical,  systematic  process  to  use  when  planning 
instruction,  together  with  the  necessary  ID  principles  and  skills  for  producing  technically  sound, 
teacher  delivered  instruction.  Thus,  our  courses  are  designed  to  enhance  the  application  of 
instructional  design  for  teachers. 

From  the  beginning,  our  aim  has  been  to  develop  applications  of  the  instructional  design 
process  that  would  serve  the  planning  of  teacher  delivered  instruction,  rather  than  materials-based 
instruction.  We  have  never  envisioned  our  goal  as  preparing  preservice  teachers  to  be  full-fledged 
instructional  designers  in  the  schools,  but  rather  to  develop  teachers  who  can  apply  a  systematic 
process  for  developing  more  effective  instruction,  especially  where  the  teacher  is  likely  to  be  the 
centerpiece  of  instructional  delivery.  At  the  same  time  we  have  taken  the  view  that  all  of  the 
major  steps  in  a  generic  ID  model  and  the  basic  skills  associated  with  them  can  be  taught  at  the 
undergraduate  level  and  do  have  value  for  the  classroom  teacher.  Support  for  this  view  can  also 
be  found  in  writings  by  Beilby,  1974;  Stolovitch,  1980;  Dick  and  Carey,  1985;  Dick  and  Reiser, 
1989;  Martin  &  Clemente,  1990;  and  Reiser  &  Mory,  1991. 

At  UNCW  the  instructional  design  component  of  the  curriculum  is  intended  to  provide 
students  with  a  particular  way  of  thinking  about  and  planning  instruction.  We  have  attempted  to 
create  a  teacher  education  program  (not  merely  a  single  course)  that  promotes  the  adoption  and 
development  of  a  systematic  approach  to  instruction.  Consequently,  the  entire  program,  at  least 
ideally,  is  designed  to  support  the  application  of  instructional  design  skills  and  principles  to 
teaching.  Methods  instructors  and  practicum  supervisors  are  expected  to  support  the  further 
development  of  students'  ID  skills  initially  acquired  in  the  instructional  design  course.  Of  course, 
we  do  find  that  certain  ID  concepts  and  skills  are  better  understood,  receive  greater  emphasis,  and 
are  better  maintained  over  the  course  of  the  program  than  are  others.  Not  surprisingly,  those 
aspects  that  are  most  easily  transferred  to  the  day-to-day  activities  of  teachers  get  emphasized. 
In  addition  to  Gagne's  irxonomy  and  the  events  of  instruction,  objectives  receive  continued 
emphasis  (although  typically  in  abbreviated  form),  as  do  learner  analysis  and  criterion  referenced 
testing  practices. 

Purpose 

This  study  continues  to  investigate  the  effects  of  preservice  ID  skills  on  classroom 
practice.  Do  graduates  of  our  program  actually  use  the  ID  skills  we  teach  them?  If  so,  how? 
If  not,  why  not?  Are  they  modified  in  practice? 

In  particular,  the  study  focuses  on  the  relationship  between  teacher  mental  planning  and 
ID  skills  for  our  graduates.  Clemente  and  Martin  (1990)  have  outlined  a  comparison  of  ID 
components  and  teacher  mental  planning  which  attempts  to  link  ID  theory  with  classroom  practice. 
Although  much  has  been  written  about  the  teacher  planning  process  (Yinger,  1979;  Morine- 
Dershimer,  1978-79;  McCutcheon,  1980;  Clark  &  Peterson,  1986),  we  in  the  instructional  design 
field  seem  to  have  neglected  a  valuable  body  of  research.  This  study  attempts  to  not  only  show 
the  value  of  ID  skills  for  preservice  teachers,  but  also  link  those  skills  with  teacher  mental 
planning. 

Process 

We  identified  recent  graduates  (1980-1990)  from  our  undergraduate  preservice  teacher 
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education  programs.  We  asked  them  to  respond  to  a  four-part  survey  which  covered 
demographics,  general  information,  yearly  planning,  unit  planning,  and  daily  planning.  Similar 
questions  addressed  each  aspect  of  planning.  A  randomly-selected  group  received  two  surveys 
with  instructions  to  share  one  with  colleagues  who  had  not  graduated  from  UNCW.  The  survey 
was  adapted  from  instruments  developed  by  Barbara  Martin  and  Robert  Reiser. 

The  second  phase  of  the  study  involved  interviewing  teachers  in  more  detail  to  flesh  out 
the  initial  findings  from  the  survey  and  to  explore  further  the  process  of  mental  planning. 
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Phase  I:  Initial  Findings  and  Opportunities  for  Reflection 

A  review  of  t'.e  data  has  resulted  in  the  following  observations.  Tables  1-8  provide  a 
more  detailed  summary  of  the  survey  responses.  These  areas  are  explored  farther  in  the  follow-up 
interview  phase  of  this  study. 

1 .  8 1  %  of  teachers  felt  that  a  knowledge  of  ID  processes  had  improved  their  planning  (6  % 
No;  13%  Not  Sure). 

2*  The  crucial  elements  of  the  ID  process  were  goals,  learner  analysis,  objectives,  tests, 
activities/strategies,  and  revision  of  instruction.  These  were  also  the  areas  treated  formally  by 
teacher?.  Those  elements  considered  helpful  (if  time  allowed)  were  task  analysis,  classification 
of  learnings,  instructional  plans,  and  trying  out  instruction.  These  were  treated  informally  and 
usually  implemented  through  mental  planning.  Of  particular  interest  was  the  fact  that  instructional 
plans  were  found  to  be  helpful  but  not  crucial. 

3.  Written  plans  focused  more  on  objectives  and  tests  and  to  a  lesser  degree  on 
activities... but  not  on  instructional  plans.  It  appears  that  use  of  written  plans  may  be  affected  by 
administrative  requirements. 

4.  Unit  planning  concentrated  on  goals,  learners,  objectives,  tests,  and  strategies.  Daily 
plans  emphasized  learners,  objectives,  strategies/activities,  instructional  plans,  and  revision  of 
instruction.  Objectives  were  also  stressed  at  the  yearly  level.  Of  particular  interest,  and  perhaps 
expected,  was  the  low  response  to  trying  out  instruction... treated  informally  and  mentally  and 
considered  helpful  if  time  allowed. 

5.  Although  teachers  regarded  mental  and  written  planning  as  almost  equal  in  importance 
at  the  yearly,  unit,  and  daily  levels,  they  favored  mental  planning  overall. 

6.  Teachers  deviated  more  from  yearly  plans  and  less  from  unit  and  daily  plans.  Daily  and 
unit  planning  were  viewed  as  more  important. 

7.  66%  of  teachers  indicated  that  more  than  50%  of  their  unit  planning  resulted  in  written 
plans.  47%  responded  that  they  wrote  more  than  50%  of  their  daily  plans. 

8.  Content  took  more  time  at  the  yearly  level  and  least  at  the  daily  level.  Slightly  more  time 
was  given  to  materials  in  daily  planning.  More  time  was  spent  on  activities  at  the  unit  and  daily 
level.  Tests  took  equal  time  at  all  levels.  More  time  was  allocated  to  objectives  at  the  yearly 
level. 


Phase  II:  Spontaneously  Systematic  or  Systematically  Spontaneous? 

The  following  comments  were  gathered  from  in-depth  interviews  with  teachers  in  which  we 
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discussed  their  planning  before,  during,  and  after  the  school  year.  I  have  grouped  representative 
quotes  into  three  major  areas-time  frames  of  planning,  the  framework  or  systematic  approach  to 
planning,  and  the  mental  processes  of  planning. 

Time  Frames. 

Teachers  do  indeed  plan  in  different  time  frames  with  varying  emphases  at  each  level. 

1.  Yearly  Planning 

•    I  sit  down  at  the  beginning  of  the  year  and  ask  *what  do  I  want  to  accomplish?' 

•  I  plan  for  the  year  in  chunks,  correlating  with  the  science  and  social  studies  teacher. 

•  The  timelines  are  in  my  minu... basically  mental.. .1  don't  write  down  the  scope  and 
sequence. 

•  I  think  back  on  what  worked  last  year  and  what  didn't. 

2.  Unit  Planning 

•  I  map  it  out  in  home  base... with  monthly  themes  and  activities. 

•  A  monthly  calendar  on  PrintShop...a  packet  with  objectives,  times,  material  lists, 
activities,  and  assessment  or  evaluation  ideas. 

3.  Daily  Planning 

•  A  notebook  with  a  lesson  outline  including. .  .objective,  focus  and  review,  teacher  input, 
guided  practice,  independent  practice,  and  closure. 

•  I  like  to  see  tour  weeks  at  a  glance... in  each  daily  block  I  use  shorthand... a  rough 
sketch  of  what  I  want  to  do... not  a  lot  of  detail. 

•  Every  class  is  at  a  different  point...notecards...a  record  of  where  they're  at.  There's 
a  lesson  plan  laid  out... this  is  where  I  want  to  be  and  want  to  do. 

Systematic  Framework 

There  is  adequate  evidence  that  teachers  approach  their  planning  in  very  systematic  ways, 
emphasizing  at  least  the  following  ID  skills. 

1.       Learner  Analysis 

•  We  really  plan  as  to  how  W3  see  the  needs  of  the  children. 

•  The  reactions  of  the  students  actually  helped  me  in  my  planning  for  the  next  time  I 
taught  this. 

•  I  try  to  know  my  kids,  their  minds,  who  they  are.   Test  scores  are  available  for 
reference. 

•  I  look  at  the  children  and  try  to  understand  where  they're  coming  from. 

•  I  don't  like  to  have  a  preconceived  idea... although  I  do  want  to  know  if  there  are 
problems  in  their  lives. 

•  I  use  their  cumulative  folders  and  sometimes  talk  to  other  teachers,  especially  to  see 
what  worked  well  for  the  child  in  other  classes. 

•  I've  got  to  watch  MTV,  read  the  newspaper,  do  all  this  stuff  to  bring  in  what's  really 
relevant  and  fun  to  these  kids,  or  I'm  going  to  lose  them. 
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2. 


Objectives 


•  I  use  the  list  of  competency  goals  and  indicators  as  a  checklist. .  .as  we  accomplish  them 
it's  a  guide  to  progress. 

•  Objectives  and  goals  drive  my  curriculum. 

•  I  tell  them  everything  has  a  purpose,  everything  has  a  goal,  everything  has  an  objective 
and  an  ultimate  end  to  it  and  you  have  to  know  what  that  is  when  you  go  through  the 
whole  process. 

3.  Tes'i 

•  I  set  out  with  the  goal  of  what  I  want  the  students  to  achieve  and  learn. .  .don't  make  out 
the  test  till  maybe  the  third  week  of  the  unit... want  to  see  what  concepts  are  being 
grasped... to  see  how  comfortable  and  competent  they  are. 

•  I  have  to  know  if  I've  given  them  enough  practice.  What  is  the  best  item  format  for 
them? 

•  I  can  measure  their  performance  only  if  there's  been  sufficient  instruction  and  practice. 

4.  Instructional  Strategies 

•  I  brake  for  the  teachable  moment... I  am  spontaneous... use  ideas  on  the  spur  of  the 
moment. 

•  Most  of  my  planning  time  I'm  working  on  how  to  get  things  across  to  the  kids. 

•  When  the  kids  begin  telling  you  what  they'd  like  to  do... when  they  understand  the 
purpose... then  you  can  really  start  envisioning. 

•  The  content  varies  but  the  system  or  framework  is  constant. 

•  I  follow  a  six  step  lessor;  plan-objectives,  focus  and  review,  teacher  input,  guided 
practice,  independent  practice,  and  closure... this  is  my  framework. 

•  I  use  different  materials  depending  upon  current  events,  but  the  concepts  and  objectives 
remain  the  same. 

5.  Formative  Evaluation 

•  >  don't  keep  my  lesson  plans  from  year  to  year.  It  really  changes.  Every  year's 
different-my  kids  are  different-what's  going  on  is  different-the  world  is  different. 

•  I  think  about  what  worked  and  what  didn't. 

The  Process:  Mental  Imagery 

The  following  statements  by  teachers  certainly  support  McCutcheon's  funding  that  "much  of 
[teaching  planning!  never  appeared  on  paper... [but]  resembled  a  rehearsal  of  the  lesson,  an 
envisioning  of  what  would  happen."  (1980,  p.  7) 

•  I  visualize  my  classroom... mentally  see  it. ..like  playing  a  videotape. 

•  I  had  to  get  it  down  on  paper  while  I  had  it  envisioned. 

•  We're  thinking  these  things  in  our  minds... seeing  them. 

•  I  don't  get  much  sleep  in  August.  I  envision.. .I'm  designing  my  classroom... doing  a 
bulletin  board... teaching  a  lesson.  It's  visual.  I  can  actually  see  it  in  my  mind. 

•  If  I've  envisioned  it-how  it's  going  to  go-it's  almost  as  if  I've  done  it  before-I'm 
trying  to  relive  what  I've  seen. 

•  If  there's  a  glitch  in  the  lesson,  it's  like  static  on  the  video. 

•  I  "see"  potential  problems  as  well-I  look  at  the  best  and  the  worst. 

•  Even  while  I'm  teaching... just  like  something  jumps  out  at  you... I  can  see  what  I'm 
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going  to  do. 

•  Sometimes  I  don't  write  lesson  plans... but  my  classes  were  clicking... I  had  the  plans 
in  my  mind... It  worked  because  of  my  internal  framework. 

•  I  like  to  see  things  systematically  done  and  I'm  going  over  them  in  my  mind,  putting 
all  the  visual  pieces  together. 

A  Few  Concluding  Thoughts 

This  is  exciting  stuff-bridging  the  gap  between  theory  and  practice.  This  is  where  our  future  lies 
as  instructional  designerc-devel oping  collegial  relationships  with  front  line  practitioners.  We  can 
help  teachers  apply  ID  skills  because,  as  one  i&cher  observed:  "Once  we  get  the  framework,  the 
system,  we're  constantly  using  it  and  revising  it."  This  is  where  we  can  really  make  a  difference 
in  transforming  education,  in  ensuring  successful  performances  by  our  students,  in  meeting  the 
challenge  to  break  out  of  the  mediocrity  cycle. 
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Table  1:  Formal  and  Informal  Use  of  ID  Processes 


Formal  Informal 

a.  develop  or  review  course  and         67%  33% 
unit  goals 

b.  develop  a  task  analysis  or  33%  67% 
learning  hierarchy  to  identify 

prerequisite  skills  and  sequence 

c.  classify  types  of  learning  17%  83% 
indicated  in  the  content 

d.  analyze  the  abilities  and  75%  75% 
needs  of  learners 

e.  develop  performance  75  %  25  % 
and/or  behavioral  objectives 

f.  develop  tests  that  match  67%  33% 
the  learnings  described  in  the 

objectives 

g.  select  or  produce  learning  71%  29% 
activities  and  strategies  that 

match  the  type  of  learning  and 
objective 


h.  follow  a  systematic  instruc-  60%  40% 
ional  plan  (e.g.,  Gagne's  events 

of  instruction,  the  N.C.  six 
point  p1on,  or  Madeline  Hunter's 
steps,  .  ..) 

i.  try  out  the  instruction  prior  8%  92% 
to  using  it  in  the  classroom 

j.  revise  the  instruction  b.^sed  53%  47% 

on  the  results  observed  during 

teaching 
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Table  2:  Written  or  Mental  Plans 


Written  Mental 

a.  develop  or  review  course  and  56%  44% 
unit  goals 

b.  develop  a  task  analysis  or  29%  71% 
learning  hierarchy  to  identify 

prerequisite  skills  and  sequence 

c.  classify  types  of  learning  19%  81% 
indicated  in  the  content 

d.  analyze  the  abilities  and  39%  61% 
needs  of  learners 

e.  develop  performance  74%  26% 
and/or  behavioral  objectives 

f.  develop  tests  that  match  76%  24% 
the  learnings  descrioed  in  the 

objectives 

g.  select  or  produce  learning  60%  40% 
activities  and  strategies  that 

match  the  type  of  learning  and 
objective 

h.  follow  a  systematic  instruc-  53%  47% 
tional  plan  (e.g.,  Gagne's  events 

of  instruction,  the  N.C.  six 
point  plan,  or  Madeline  Hunter's 
steps,  etc.) 

i.  try  out  the  instruction  prior  14%  86% 
to  using  it  in  the  classroom 

j .  revise  the  instruction  based  55  %  45  % 

on  the  results  observed  during 

teaching 


2.15 


ERIC  212 


Table  3:  Use  of  ID  Processes  in  Yearly,  Unit,  and  Daily  Planning 


a.  develop  or  review  course  and 
unit  goals 

b.  develop  a  task  analysis  or 
learning  hierarchy  to  identify 
prerequisite  skills  and  sequence 

c.  classify  types  of  learning 
indicated  in  the  content 

d.  analyze  the  abilities  and 
needs  of  learners 

e.  develop  performance 
and/or  behavioral  objectives 

f.  develop  tests  that  match 
the  learnings  described  in  the 
objectives 

g.  select  or  produce  learning 
activities  and  strategies  that 
match  the  type  of  learning  and 
objective 

h.  follow  a  systematic  instruc- 
tional plan  (e.g.,  Gagne's  events 
of  instruction,  the  N.C.  six 
point  plan,  or  Madeline  Hunter's 

steps,  etc.) 

i.  try  out  the  instruction  prior 
to  using  it  in  the  classroom 

j.  revise  the  instruction  based 
on  the  results  observed  during 
teaching 


Year  Unit  Day 

50%  70%  30% 

30%  45%  25% 

10%  25%  35% 

30%  50%  65% 

40%  45%  45% 

10%  55%  35% 

20%  55%  70% 

10%  35%  70% 

10%  20%  25% 

30%  55%  75% 
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Table  4:  The  Value  of  ID  Processes 


a.  develop  or  review  course  and 
unit  goals 

b.  develop  a  task  analysis  or 
learning  hierarchy  to  identify 
prerequisite  skills  and  sequence 

c.  classify  types  of  learning 
indicated  in  the  content 

d.  analyze  the  abilities  and 
needs  of  learners 

e.  develop  performance 
and/or  behavioral  objectives 

f.  develop  tests  that  match 
the  learnings  described  in  the 
objectives 

g.  select  or  produce  learning 
activities  and  strategies  that 
match  the  type  of  learning  and 
objective 

h.  follow  a  systematic  instruc- 
tional plan  (e.g.,  Gagne's  events 
of  instruction,  the  N.C.  six 
point  plan,  or  Madeline  Hunter's 
steps,  etc.) 

i.  try  out  the  instruction  prior 
to  using  it  in  the  classroom 

j.  revise  the  instruction  based 
on  the  results  observed  during 
teaching 


Crucial  Help-  Uhtmpcr- 

ful  tant 

67%  33%  0% 

25%  67%  8% 

17%  50%  33% 

92%  89?  0% 

83%  0%  17% 

75%  17%  8% 

83%  17%  0% 

17%  75%  8% 

0%  75%  25% 

67%  33%  0% 
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Table  5:  Importance  of  Written  and  Mental  Plans 


Written 

Mental 

Equal 

Overall 

25% 

37.5% 

37.5% 

Year 

31% 

31% 

38% 

Unit 

29% 

7% 

64% 

Daily 

33% 

27% 

40% 

Table  6:  Following  Plans 


Very  Closely 
Closely 

(<  25%  deviation) 


(25-49%  deviation) 


(50-75%  deviation) 

Very  Loosely 

(>  75%  deviation) 


Year 

Unit 

Daily 

8% 

8% 

13% 

46% 

77% 

67% 

31% 

15% 

20% 

15% 

0% 

0% 

0% 

0% 

0% 
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Table  7:  Importance  of  Planning 

Year  Unit  Daily 

Crucial  23%  31%  60% 

(100%  of  the  time) 

Useful  46%  69%  40% 

(75%  of  the  time) 

Generally  Useful  31%  0%  0% 

(50%  of  the  time) 

Minimally  Useful  0%  0%  0% 

(25%  of  the  time) 

Not  Very  Useful  0%  0%  0% 

(<  10%  of  the  time) 


Table  8:  Amount  of  Written  Planning 

Overall  Year  Unit  Daily 

More  than  75%  29%  33%  33%  27% 

50%  -  74%  18%  20%  33%  20% 

25% -49%  35%  27%  27%  33% 

Less  than  25%  18%  20%  7%  20% 
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Text  with  Compressed  Audio 


1 

Currently  there  is  a  high-paced  search  for  the  perfect  medium.  One  that 
delivers  instruction  so  that  learning  is  effective  and  efficient  Is  it  possible  that 
with  all  the  focus  on  "high -tech/1  one  viable  solution  is  being  overlooked?  Audio 
tapes  are  one  of  the  most  popular  forms  of  entertainment  we  have  today.  Just 
walking  down  the  street  you  see  many  people  "plugged  in."  Audio  is  also 
increasingly  being  used  for  instruction  because  it  is  inexpensive,  convenient,  and 
portable.  Unlike  text,  audio  provides  a  human  touch  to  instruction.  The  voice  can 
motivate  the  learner  through  the  enthusiasm  expressed  for  the  subject. 

Statement  of  the  Problem 

One  problem  with  audio  is  the  time  it  takes  to  listen  to  a  program.  When 
involved  in  a  two-way  conversation,  the  average  rate  of  speech  is  comfortable; 
with  one-way  audio,  the  pace  seems  dreadfully  slow.  It  has  been  shown  that  the 
human  brain  can  comprehend  information  that  is  d  'ivered  nearly  twice  as  fast 
as  the  original  speed  of  delivery  (Carver,  1982).  Through  the  use  of  speech 
compression  technology,  it  is  possible  to  accelerate  a  recorded  voice  without 
significant  distortion  of  the  pitch  or  natural  quality  (Olson,  1985). 

Another  problem  associated  with  audio  programs  is  that  they  are  just  taped 
versions  of  books,  lectures,  or  seminars,  and  like  extemporaneous  conversation 
may  contain  information  that  is  extraneous.  It  has  been  shown  that  the 
effectiveness  of  instructional  materials  can  be  increased  by  employing  a 
systematic  design  process  that  removes  irrelevant  infon  lation  (Mengel,  1982). 

Augmenting  systematically  designed  text  with  audio  may  help  increase 
learners'  attention  to  the  material  because  of  it's  steadily  pac^d,  non-stop 
information  flow.  By  using  audio  along  with  text,  learners  maintain  the  benefit  of 
visual  design  features.    It  is  possible,  since  systematically  designed  materials 
h.  ve  a  high  density  of  information,  compressed  speech  technology  may  "over- 
compress"  the  instruction.  All  that  may  be  needed  is  a  normal  speech  audio  tape  for 
pacing.  Therefore,  this  study  compared  two  forms  of  audio  augmentation,  normal 
and  compressed  speech. 

Cognitive  Theory 

Most  individuals  have  sufficient  semantic  cognitive  capacity  to  carry  on  a 
conversation  at  a  rate  of  125  to  150  wpm  find  responding  at  about  the  same  rate 
internally  until  the  response  is  voiced.  If  added  together,  one  might  theorize  that 
the  average  semantic  cognitive  capacity  would  be  about  250-300  wpm.  If  involved 
in  a  one-way  semantic  communication  such  as  reading,  or  listening,  the 
cognitive  capacity  should  remain  the  same.  Increasing  the  input  level  to  250  to  300 
wpm  through  the  use  of  compressed  speech  should  help  keep  attention  levels  high 
and  optimize  learning. 

Research  has  shown  that  the  average  reading  level  of  most  Americans  is 
between  250-300  wpm,  with  the  average  college  student  reading  at  about  280  wpm 
(Taylor,  1965).  Humans  can  successfully  listen  at  rates  of  250-300  wpm  using 
speech  compression  technology.  Carver  (1981),  refers  to  this  optimal  efficiency 
rate  (300  wpm)  of  reading  or  listening  (auding)  as  the  "rauding  rate."  This  rate  is 
highly  efficientf  but  somewhat  less  effective  than  lower  rates  when  tested  using 
traditionally  developed  Earning  materials.  Effective  comprehension  of  this  type 
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of  material  begins  to  decline  around  250  wpm  using  compressed  speech  (Foulke, 
1966). 

The  evidence  strongly  indicates  that  the  cognitive  capacity  of  the  conscious 
working  memory  is  limited.  Input  from  the  five  senses  fill  it  to  the  limit  moment 
by  moment.  As  the  working  memory  is  filled,  information  is  moved  to  long-term 
memory  or  is  lost.  Humans  are  constantly  being  bombarded  by  biological, 
physiological,  and  psychological  input,  which  takes  up  this  limited  cognitive 
capacity.  To  some  extent,  humans  determine  what  is  placed  in  the  working 
memory  through  a  process  known  as  selective  perception.  What  we  choose  to 
perceive  often  relates  to  what  Gagne  (1985)  terms  executive  controls,  made  up  of  th^ 
sum  of  our  past  experience  and  future  expectations.  Selective  perception  provides 
attention  to  those  things  we  perceive  as  important.  Research  has  shown  that 
attention  itself  utilizes  cognitive  capacity.  Therefore,  to  optimize  learning, 
attention  to  distractions  must  be  reduced  and  working  memory  input  should  be 
limited  to  the  information  required  for  learning.  It  is  also  desirable  to  fill  the 
cognitive  capacity  with  the  learning  stimulus  at  an  optimum  rate  so  that 
distinctive  thoughts  do  not  have  time  to  enter  the  working  memory. 

Examination  of  the  research  (Mengel,  1982)  shows  that  systematically 
designed  materials  have  a  higher  concept  density  than  traditionally  developed 
materials.  In  effect,  the  instruction  is  already  compressed  to  some  extent.  It  is 
possible  that  this  would  increase  the  rate  of  concepts  flowing  into  the  working 
memory  requiring  less  compression  from  another  source  to  total  the  cognitive 
capacity.  In  fact,  designing  instruction  (increasing  density)  could  create  too 
dense  an  information  flow  at  rates  suggested  by  past  research. 

Since  the  process  of  attention  (or  selective  perception)  itself  uses  up  vital 
capacity,  reducing  the  need  for  attention  by  systematically  designing  instruction 
may  provide  even  more  capacity  for  learning.  Berlyne  (1960)  suggests  that  a 
higher  degree  of  involvement  with  learning  materials  should  correlate  with  a 
higher  degree  of  recall. 

Systematically  designed  instruction  is  objectively  based  so  that  all  of  the 
information  is  aimed  toward  mastering  the  objectives.  Practice  actively  involves 
the  student  in  the  learning  process.  In  addition,  redundant  information  presented 
in  two  channels  (reading  and  listening)  and  using  two  modes  (linguistic  and 
iconic)  should  create  an  even  higher  degree  of  involvement  with  the  materials, 
thus  increasing  learning  (Bradtmueller,  1978;  Nugent,  1982).  If  the  need  for 
attention  is  further  reduced  by  providing  an  environment  with  a  low  level  of  noise, 
learning  should  be  increased  even  more  (Hsia,  1968).  By  combining  these 
methods  the  involvement  with  the  learning  materials  should  be  at  a  maximum 
level  and  the  effectiveness  of  higher  compression  rates  should  be  increased. 

Measuring  Compression  Rates 

Research  has  identified  several  methods  of  measuring  compression  rates. 
Many  studies  use  a  percentage  of  compression  with  the  original  tape  defined  as  0% 
and  the  experimental  tapes  from  25%  to  100%  compression.  Other  studies  define 
the  original  tape  as  100%  and  the  experimental  tapes  as  195%  to  200%  compression. 
The  problem  with  both  percentage  methods  is  that  the  spe&d  of  the  original  tape  is 
usually  not  stated. 

Conversational  speed  has  been  measured  at  125-150  wpm.  Professional 
voices  have  been  identified  as  averaging  175  wpm.  25%  (tr  125%)  compression  of 
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125  wpm  equals  156.25  wpm,  which  is  barely  out  of  the  conversational  speech 
range.  75%  (or  175%)  compression  of  125  wpm  equals  a  25%  (or  125%)  compressir  i 
of  175  wpm  which  is  218.79  wpm.   These  large  differences  make  the  studies 
difficult  to  interpret.  What  does  1.5  times  or  2  times  normal  speed  mean? 

Other  studies  do  identify  the  exact  wpm  used  (Boyle,  1969),  but  don  t  always 
clearly  state  the  rationale  behind  their  choices  of  experimental  speeds.  It  would 
seem  that  in  early  research  it  was  necessary  to  experiment  with  a  variety  of 
speeds;  with  a  more  solid  research  base,  compression  rates  can  now  be  deliberately 
chosen. 

Bradtmueller  (1978)  states  that  "It  is  theoretically  possible  to  speed  up  tapes 
to  any  rate  but  the  human  hearing  mechanism  begins  to  have  difficulty  processing 
messages  somewhere  between  300  and  500  words  per  minute"  (p.  4).  Acceptable 
comprehension  occurs  up  to  300  words  per  minute  and  is  most  efficient  at  that  level 
(Carver,  1982;  Sticht,  1968).  Given  discretionary  control  over  the  rate  of 
presentation,  students  prefer  to  work  at  a  rate  around  225  wpm,  well  below  the 
limits  of  their  capabilities  (Lass  et  al.,  1974;  Foulke  &  Sticht,  1967). 

The  four  rates  have  appeared  in  the  research  that  are  of  interest: 

175  wpm  -  This  is  the  approximate  rate  reportedly  used  by  professionally 
trained  readers  (Foulke,  1966).  It  is  still  within  the  normal  speech  range.  Media 
sophistication  has  probably  acclimated  most  people  to  this  rate  of  speech.  Rates 
below  this  level  were  not  chosen  or  ranked  very  highly  by  subjects  (Lass  et  al, 
1974;  Orr  et  al.,  1969).  Slow  speakers  generate  negative  evaluations  by  the 
listeners,  whereas  fast  speakers  are  described  as  fluent,  persuasive,  and  credible 
(Apple,  Streeter,  and  Krauss,  1979;  Miller  et  al.,  1976). 

225  wpm  -  This  is  the  rate  reported  to  be  preferred  by  learners  (Lass  et  al., 
1974;  Foulke  &  Sticht,  1967;  Orr  et  al.,  1969).  At  this  rate  learners  are  working 
below  their  abilities  (Foulke  &  Sticht,  1967).  It  is  29%  compression  of  175  wpm. 

262  wpm  -  This  rate  is  a  half-way  point  between  225  wpm  and  300  wpm.  It  is 
also  a  50%  compression  level  for  175  wpm.  In  a  study  of  radio  advertising  using 
professionally  made  recordings,  LaBarbera  and  MacLachlan  (1979)  state:  "As 
long  as  the  speech  is  not  speeded  up  by  more  than  50%  the  listener  will  be  unaware 
that  there  has  been  an  electronic  alteration  of  the  original  recording"  (p.  30). 
Some  studies  show  (Foulke,  1966)  comprehension  of  traditional  materials  begins 
to  decline  somewhere  between  250  wpm  and  300  wpm. 

300  wpm  -  This  rate  consistently  appears  in  the  research  as  the  top  end  level 
for  efficient  comprehension  (Carver,  1982),  although  subsequent  recall  may  not  be 
as  great  as  it  is  for  lower  levels  of  speed.  A  large  number  of  studies  show  that  rates 
beyond  this  level  cause  comprehension  to  drop  off  precipitously  (Carver,  1982; 
Sticht,  1968). 

For  this  study,  175  wpm  was  selected  as  the  speed  of  the  original  recording, 
and  defined  as  1  normal"  speech.  For  the  compressed  speech  recording,  262  wpm 
was  chosen.  Since  this  rate  approaches  the  cognitive  capacity  limit  of  280-300,  it 
was  believed  that  it  would  not  push  slower  students  past  their  personal  limits. 
Higher  rates  were  not  chosen  because  comprehension  may  have  been  sacrificed. 

Purpose  &  Hypotheses 

The  purpose  of  this  research  was  to  determine  if  systematically  designed 
text  augmented  with  compressed  speech  could  increase  the  number  of  objectives 
achieved  and  reduce  the  amount  of  learning  time  needed  for  mastery  of  the 
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objectives.  The  effectiveness  of  learning  was  defined  as  the  number  of  objectives 
mastered.  Efficiency  was  defined  as  the  effectiveness  divided  by  instructional 
time  multiplied  by  100.  A  self-report  method  was  used  to  reveal  the  actual 
instructional  time  elapsed.  This  allowed  investigation  of  the  actual  time  the 
students  used  to  complete  the  instruction,  including  rewinding,  practice  activities, 
and  review. 

It  was  hypothesized  that:  1.  The  text  materials  augmented  with  compressed 
speech  audio  tapes  would  be  ,uore  effective  than  text  alone.    2.  The  text  materials 
augmented  with  compressed  speech  audio  tapes  would  be  more  efficient  (concepts 
learned  per  minute)  than  either  text  alone  or  text  augmented  with  normal  speech 
audio  tapes. 

Method 

The  methodology  was  designed  to  determine  the  optimal  type  of  audio 
augmentation  for  systematically  designed  text  materials.  The  optimal 
augmentation  was  defined  as  the  one  that  allowed  for  the  highest  comprehension 
and  the  largest  time  savings. 

Subjects 

The  participants  in  this  study  were  78  students  from  5  schools  with 
vocational  education  programs  in  the  State  of  Florida.  Urban  and  rural  schools 
were  both  included.  23  program  areas  were  represented.  There  were  28  males  and 
50  females  of  which  54  were  white  and  24  non-white.  The  ages  ranged  from  16  to  67 
years. 

The  participants  had  reading  levels  that  ranged  from  the  3.1  to  12.9  grade 
level  according  to  their  total  reading  scores  on  the  Tost  of  Adult  Basic  Education 
(TABE).  The  mean  reading  score  was  6.8;  the  standard  deviation  was  2.0. 
Subjects  were  randomly  assigned  to  the  control  group  and  the  treatment  conditions 
by  placing  them  in  groups  in  rotating  order  as  they  entered  into  the  lab. 

Materials 

The  instructional  materials  selected  were  designed  for  the  State  of  Florida 
by  the  Center  for  Instructional  Development  and  Services,  The  Florida  State 
University. 

The  E7nployability  Skills  Series  is  a  set  of  instructional  materials 
designed  to  teach  secondary  students  and  adults  the  skills  involved  in 
getting  and  keeping  a  job.  The  purpose  of  the  series  is  to  provide  educators 
with  ready-to-use  student  materials  that  are  competency-based  and 
validated  for  effectiveness  and  acceptability  in  actual  classrooms. 
Oriented  to  young  adults  but  written  at  a  reading  level  of  approximately 
fifth  grade,  the  materials  have  a  flexible  format  that  can  be  used  in  a 
variety  of  instructional  settings.  (Florida,  1979,  p.  6)  « 
This  series  was  selected  because  it  was  developed  using  a  rigorous  instructional 
design  process  including  formative  and  summative  evaluations  and  periodic 
updates  (Kromhout,  Farrow,  Foster,  &  Morse,  1978).  The  materials  were  written  at 
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a  reading  level  that  would  produce  efficient  and  effective  learning  for  the  specified 
population. 

The  unit  Good  Work  (Florida,  1979)  was  chosen  because  it  was  the  most 
widely  and  regularly  used  module.  The  total  instructional  package  included:  1. 
A  text  module  from  the  series  covering  the  introduction,  knowledge  objectives,  and 
instruction  for  thise  objectives.  %  Practice  activities  corresponding  to  the  text.  3. 
Audio  tapes  corresponding  to  the  chosen  text. 

The  text  of  the  written  materials  was  used  as  a  script  for  the  audio  tapes,  as 
suggested  by  other  studies  (Nasser  &  McEwen,  1976;  Nugent,  1982;  Rohwer  & 
Harris,  1975).  Only  the  text  portion  of  the  materials  were  recorded,  the  practice 
exercises  were  not.  A  male  voice  was  selected  to  read  the  text  (Foulke  &  Sticht, 
1967).  This  narrator  was  a  professional  radio  announcer.  Additionally,  to 
enhance  quality,  the  scenarios  pictured  in  cartoons  or  described  in  the  text  were 
read  by  a  variety  of  male  and  female  character  voices.  The  recording  was  made  in 
a  professional  sound  studio. 

In  order  to  record  the  tape  at  an  average  speed  of  175  wpm  the  narrator  used  a 
script  with  175  word  segments  marked  off  and  used  a  stop  watch  to  pace  the  segment 
to  a  minute.  After  the  audio  tapes  were  recorded  at  normal  speed,  copies  were  made 
at  the  compression  speed  using  a  Model  H910  Eventide  Harmonizer/Pitch  Shifter. 
The  same  technique  was  used  to  pace  the  compressed  speech  tape  where  262  word 
segments  wen  marked  off  and  the  speech  was  digitally  compressed  to  complete 
each  segment  n  one  minute.  Music  and  tone  signals  for  page  turning  were  added 
after  the  compressed  copies  were  made  so  that  the  music  would  no4  sound  distorted 
and  the  tone  would  not  be  too  brief.  Music  was  used  only  at  the  beginning  and  end  of 
each  lesson. 

Variables 

The  first  dependent  variable  was  effectiveness  defined  as  the  mastery  of 
knowledge  objectives  as  measured  by  open-ended  tests  that  asked  students  to  list,  to 
identify,  and  to  name.  Mastery  is  defined  in  the  answer  keys  provided  in  the 
instructor  s  guide  of  the  learning  materials.  The  tests  had  already  been  validated 
during  the  development  of  the  instructional  materials  to  insure  then  ability  to 
measure  the  objecth  3S  (Kromhout  et  al.,  1978).  A  mastery  level  of  60%  was  used  for 
field  testing  the  mate,  ;als,  and  for  the  purpose  of  determining  mastery.  If  the  tests 
had  been  altered  to  a  multiple  choice  for  ease  of  scoring,  the  validity  of  the  tests 
might  have  been  altered. 

The  second  dependent  variable  was  efficiency  defied  as  effectiveness 
divided  by  instructional  time  multiplied  by  100.  Instructional  time  was  measured 
by  a  self-report  method  of  beginning  and  ending  times  for  each  of  the  four  lessons 
in  the  instructional  materials.  (Mengel,  1982) 

The  independent  variable  was  the  augmentation  of  the  text  materials.  A 
control  group  received  only  the  text  materials  with  no  augmentation.  One 
treatment  group  received  the  text  with  a  normal  speech  audio  tape  of  175  wpm.  The 
other  treatment  group  received  a  compressed  speech  audio  tape  of  262  wpm. 

Analysis 

The  data  were  analyzed  using  a  one-way  analysis  of  variance  for  each  of 
tho  dependent  variables,  effectiveness  (mastery  of  objectives),  and  efficiency 
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(effectiveness  per  unit  of  time).  Three  groups  were  examined:  a  control  group  with 
text  only;  and  two  treatment  conditions,  text  with  normal  speech  and  text  with 
compressed  speech.  Reading  scores  for  the  Test  of  Adult  Basic  Education  (TABE) 
were  used  as  a  covariate  measure.  The  alpha  level  was  set  at  .05  using  a  two  tailed 
test.  A  sample  of  21  subjects  per  treatment  group  (63  total),  would  generate  a  power 
of  .80  where  the  alpha  level  was  .05  and  the  desired  effect  size  (the  minimum 
difference  considered  to  be  of  practical  importance)  was  .40  standard  deviation, 
(Cohen,  1977,  p.  384).  Data  were  collected  from  83  participants  in  six  schools.  One 
school  was  dropped  because  protocol  was  not  maintained.  The  final  sample 
consisted  of  a  control  group  containing  28  subjects,  a  normal  speech  group 
containing  28  subjects,  and  a  compressed  speech  group  with  22  subjects.  The 
sample  totaled  78  subjects. 

Procedure 

The  learning  environment  A  learning  laboratory  was  used  for  this  study 
to  enhance  the  ability  to  examine  the  augmentation  levels  by  reducing  error  caused 
by  noise.  Students  ustd  individual  tape  recorders  with  headphones  and  were 
allowed  to  work  at  their  own  pace. 

The  experimental  procedure.  Before  the  study  was  conducted,  management 
packages  were  produced  to  ensure  consistency  in  the  data  collection.  The  use  of  the 
packages  was  demonstrated  in  a  site  visit  to  each  school.  An  accordion  file  folder 
was  labeled  and  organized  with  management  information,  student  documents, 
and  the  audio  materials.  The  file  also  provided  space  for  forms  in  progress  and 
completed  forms.  Supervisory  materials  were  located  in  a  large  envelope  held  in 
the  front  of  the  package.  All  paperwork  was  color-coded  for  ease  of  handling. 

The  unit  was  used  in  the  same  manner  as  other  instructional  materials  in 
the  media  center  to  provide  a  true  picture  of  how  management  and  setting  might 
impact  the  use  of  compressed  speech.  For  example,  students  were  able  to  choose  the 
amount  of  the  module  they  would  complete  at  each  sitting.  The  module  was  divided 
into  four  10-20  minute  lessons  over  the  eight  objectives  to  provide  convenient 
stopping  places.  The  students  were  also  able  to  choose  their  own  schedule  for  taking 
the  lessons  and  the  post-test.  The  range  was  from  10  to  15  days.  Through  a  self- 
report  method,  the  participants  kept  track  of  the  actual  times  spent  on  each  lesson. 

The  materials  contained  an  introduction  and  several  units  of  instruction. 
According  to  the  literature  a  slight  practice  listening  to  compressed  speech  is 
enough  to  acclimate  the  student  to  the  speed  of  the  tape  (Perry,  1970).  For  this  reason, 
the  introduction  found  in  the  text  materials  was  used  on  the  audio  tapes.  The 
introduction  on  both  normal  and  compressed  speech  audio  tapes  described  the 
purpose  of  the  audio  tape  and  informed  the  learner  that  the  voice  of  the  speaker  may 
have  sounded  faster  than  normal  (Williams  et  al.,  1983-84). 

Timing  started  with  the  first  lesson.  The  subjects  were  allowed  to  replay  or 
re-read  the  introduction  as  many  times  as  they  felt  necessary.  The  students  were 
requested  to  try  to  complete  all  of  the  instruction  in  a  lesson  before  taking  a  break. 
They  were  informed  that  if  they  needed  to  stop  the  instruction  for  any  reason,  they 
were  to  write  down  the  time  they  stopped  and  then  restart  the  instruction.  The  audio 
introduction  informed  the  learners  that  they  could  rewind  the  tape  if  necessary  to 
repeat  portions  they  hadn't  u  nderstood.  This  allowed  the  researcher  to  investigate 
the  actual  timing  of  the  use  of  the  tapes  in  a  media  center.  If  the  participants  needed 
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to  rewind  the  compressed  speech  tapes  more  often,  the  analysis  would  show  that 
these  rates  are  not  as  efficient  for  learning. 

The  project  monitors  retrieved  the  instructional  packages  and  information 
sheets  from  the  subjects  and  gave  them  the  posttests  when  all  the  lessons  had  been 
completed.  The  project  monitors  were  asked  to  make  sure  that  all  of  the  blanks 
were  filled  in  on  both  the  information  form  and  the  tests.  The  tests  were  scored 
using  information  in  the  instructor's  guide  and  rechecked  later  for  consistency  in 
scoring. 

Results 

Two  separate  one-way  analyses  of  covariance  (ANCOVA)  were  used  to  test 
the  statistical  significance  of  the  hypotheses.  Reading  scores  from  the  Test  of  Adult 
Basic  Education  (TABE)  were  used  as  a  covariate  measure.  Alpha  was  pre-set  at 
.05  for  both  analyses.  The  total  n  a  78. 

First,  the  internal  consistency  of  the  comprehension  posttest  was  evaluated 
for  the  sample.  There  were  no  comparison  statistics  from  the  original  field-test 
since  the  objective  was  to  create  a  criterion-referenced  measure  that  worked  with 
80%  of  the  population.  This  would  have  necessarily  biased  any  internal 
consistency  statistics.  Since  the  purpose  of  this  study  was  to  use  the  information  in 
a  norm-referenced  context,  an  internal  consistency  reliability  coefficient, 
(Cronbach's  alpha)  was  calculated.  The  comprehension  posttest  had  a  reliability 
coefficient  of  alpha  ■  .65.  This  result  was  considered  acceptable  due  to  the  original 
objective  of  the  measurement  instrument  and  the  small  number  of  items 
(n  =  8)  on  the  test 

Cochran's  C  statistic  was  used  to  check  for  homogeneity  of  variance  for  both 
hypotheses.  There  were  no  significant  differences  among  the  variances  of  the 
treatment  groups  on  either  dependent  variable.  There  was  a  significant  difference 
for  the  time  variable  used  in  a  post  hoc  analysis.  (Table  1) 

TABLE  1 


Cochran's  C  Test 
for  Homogeneity  of  Variance 


Dependent 
Variable 

Cochran's 
C 

#  of 
Variances 

df 

Probability 

Effectiveness 

.431 

3 

27 

.312 

Efficiency 

.427 

3 

27 

.343 

Time 

.533 

3 

27 

.020* 

♦Significant  at  ft<.05. 


A  test  for  homogeneity  of  regression  was  used  to  determine  whether  the 
TABE  scores  could  be  used  successfully  as  a  covariate  measure.  No  significant 
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differences  were  found  among  the  regression  line  slopes  of  the  covariate  on  each 
dependent  variable  for  each  treatment  group.  An  ANOVA  for  reading  scores 
showed  no  significant  differences  between  the  means  of  the  treatment  groups. 

For  a  measure  of  the  relationship  between  the  variables  and  the  covariate, 
Pearson's  Product-Moment  Correlation  Coefficients  were  calculated.  There  was  a 
significant  correlation  between  TABE  and  efficiency!  and,  TABE  and 
effectiveness.  There  was  a  significant  correlation  also  between  time  and 
efficiency.  (Table  2) 

TABLE  2 

Pearson's  Product-Moment 
Correlation  Coefficients 


Effectiveness  Efficiency  Time 

r/p  r/p  r/p 


.45/.001**  .36/.001**  -.06/.312 

  .51/.001**  .09/.222 

    -.67/.001** 


Reading  Scores 

Effectiveness 

Efficiency 


^Significant  at  £<.001. 


Histograms  were  used  to  check  the  assumption  of  normality.  It  was  clear 
from  the  results  that  the  comprehension  posttest  to  measure  effectiveness  created  a 
ceiling  effect  in  all  groups.  When  examining  a  histogram  of  the  sample  for  the 
efficiency  measure,  a  skewness  of  1.2  was  found.  The  tail  was  toward  higher 
efficiency  score3. 

It  is  probable  that  this  tail  was  created  by  the  ceiling  effect  of  the 
comprehension  posttest  which  is  a  component  of  the  efficiency  formula.  These 
extreme  scores  represented  subjects  who  hit  the  ceiling  in  a  short  time  and  are 
considered  efficient  learners.  Since  their  scores  were  legitimate,  they  represent  a 
segment  of  the  population.  These  scores  were  not  deleted. 

It  was  decided  that  since  no  other  assumptions  were  violated  and 
considering  the  robust  nature  of  ANOVA  regarding  the  normality  assumption, 
these  scores  could  still  be  used.  However,  since  the  ceiling  effect  was  pronounced, 
and  could  possibly  bias  the  efficiency  measure,  it  was  determined  that  a  post-hoc 
analysis  of  the  time  component  should  be  examined  as  well.  The  results  of  this 
analysis  will  be  given  after  the  two  primary  analyses. 

A  one  way  ANCOVA  was  then  used  to  test  the  null  hypothesis  that  all 
treatment  groups  would  be  equivalent  with  regard  to  effectiveness.  Effectiveness 
was  defined  as:  the  number  of  correct  objectives  on  a  comprehension  posttest.  The 
null  hypothesis  could  not  be  rejected.  Table  3  presents  the  means  and  standard 
deviations  and  Table  4  summarizes  the  ANCOVA  statistics. 
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TABLE  3 

Means  and  Standard  Deviations 
Of  Effectiveness  Scores  by  Treatment  Group 
with  Reading  Scores  as  the  Covariate  Measure 


EfffictavRnms 


Treatment 
Group 

Covariate 
Mean 

Actual 
Mean 

Adjusted 
Mean 

Standard 
Deviation 

Group 

Size 

Text  Only 

9.19 

6.82 

6.70 

1.77 

23 

Normal  Speech 
with  Text 

9.03 

6.89 

6.82 

1.10 

28 

Compressed  Speech 
with  Text 

8.02 

6.32 

6.59 

1.70 

22 

Total 

8.80* 

6.71** 

6.71 

1.54 

78 

*  TABE  reading  scores  ranged  from  3.1  to  12.9. 
**Total  possible  score  »  8.00. 

TABLE  4 


Summary  of  ANCOVA 
for  Effectiveness  Scores  by  Group 
with  Reading  Scores  as  the  Covariate  Measure 


Source  of  F  Sign.      %  of 


Variation 

df 

SS 

MS 

Ratio 

ofF 

Var. 

TABE-Reading 
Comprehension 

1 

36.54 

36.54 

18.66** 

.000** 

20% 

Treatment 

2 

.66 

.33 

.17 

.845 

0% 

Residual 

74 

145.02 

1.96 

80% 

Total 

77 

182.22 

2.37 

100% 

♦♦Significant  at  d,<.001. 

Another  one  way  ANCOVA  was  used  to  test  the  null  hypothesis  that  all 
treatment  groups  would  be  equivalent  with  regard  to  efficiency.  Efficiency  was 
defined  as:  effectiveness/number  of  minutes  to  complete  instruction  X  100.  The 
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null  hypothesis  was  rejected;  significant  differences  among  the  groups  at  the  p<.05 
level  were  indicated.  Means  and  standard  deviations  are  presented  in  Table  5, 
and  ANCOVA  statistics  are  summarized  in  Table  6. 

TABLE  5 

Means  and  Standard  Deviations 
Of  Efficiency  Scores  by  Treatment  Group 
with  Reading  Scores  as  the  Covariate  Measure 


Treatment 
Group 

Covariate 
Mean 

Actual 
Mean 

Adjusted 
Mean 

Standard 
Deviation 

Group 
Size 

Text  Only 

9.19 

6.95 

6.74 

3.07 

28 

Normal  Speech 
with  Text 

9.03 

5.61 

5.48 

1.82 

28 

Compressed  Speech 
with  Text 

8.02 

6.99 

7.43 

3.08 

22 

Total 

8.80* 

6.48** 

6.48 

2.74 

78 

*  TABE  reading  scores  ranged  from  3.1  to  12.9. 
♦♦Individual  Scores  ranged  from  1.10  to  17.78. 

TABLE  6 


Summary  of  ANCOVA 
for  Efficiency  Scores  by  Group 
with  Reading  Scores  as  the  Covariate  Measure 


0 

ERIC 


Source  of 
Variation 

df 

SS 

MS 

F 

Ratio 

Sign. 
ofF 

%  of 
Var. 

TABE-Reading 
Comprehension 

1 

74.95 

74.95 

12.22*^ 

.001+* 

13% 

Treatment 

2 

48.38 

24.19 

3.94* 

.024* 

8% 

Residua] 

14 

454.03 

6.14 

79% 

Total 

77 

577.36 

7.50 

100% 

♦Significant  at  u<.001. 
♦Significant  at  p.<.05. 
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In  order  to  determine  which  pair  of  groups  were  significantly  different  on 
efficiency  scores,  post-hoc  Tukey  pairwise  comparisons  were  made.  The  results  of 
these  pairwise  comparisons  are  presented  in  Table  7. 

TABLE  7 

Post-Hoc  Pair-Comparisons 
for  Efficiency  Scores  by  Group 
with  Reading  Scores  as  the  Covariate  Measure 


Text  Normal  Speech 

Only  with  Text 

tfprob.  t/prob. 


Normal  Speech  -1.90/.06   

with  Text 

Compressed  Speech  .95/.34  2.71/.009* 
with  Text 


♦Significant  at  £<.01. 

Another  post-hoc  analysis  was  used  to  examine  the  Time  element  of  the 
efficiency  formula  because  o^the  ceiling  effect  apparent  in  the  effectiveness  portion 
of  the  formula.  Time  was  defined  as:  the  number  of  minutes  to  complete 
instruction.  Since  the  Cochran's  C  showed  significant  differences  among  the 
variances  of  the  treatment  groups  (Table  1),  the  individual  scores  were  placed  on  a 
graph  for  further  examination.  (See  Figure  1  for  the  graph.) 
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FIGURE  1 

Graph  of  Individual  Scores 
of  Time  to  Complete  Instruction 
Using  Five  Minutes  Intervals 


Number 
of 

Subjects 


Number 
of 

Subjects 


Number 
of 

Subjects 


Text  Only  n=28 


1 


25     50    75    100   125   150   175    200   225  250   275  300 


Normal  Speech 
with  Text  n=28 


J  I 


25     50     75    100   125   150   175    200   225  250   275  300 


Compressed  Speech 
with  Text  n=22 


)     25     50     75    100   125   150   175    200   225  250   275  300 


An  ANCOVA  was  used  although  the  assumption  of  homogeneity  of  variance 
was  violated.  It  was  determined  that  the  loss  of  power  due  to  this  violation  was  at  an 
acceptable  level.  The  ANCOVA  demonstrated  significant  differences  at  the  n  <.05 
level.  See  Table  8  for  means  and  standard  deviations  and  Table  9  for  ANCOVA 
statistics. 
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TABLE  8 

and  Standard  Deviations 
\n  Time  Scores  by  Treatment  Group 
w*t;i  Reading  Scores  as  the  Covariate  Measure 


Time 

Tr  ."ohnent 
Group 

Covj/fiate 

Mean 

Actual 

Mean 

Adjusted 
Mean 

Standard 
Deviation 

Group 
Size 

Text  Only 

sU9 

112.50 

113.54 

49.45 

28 

Normal  S>?ech 
with  Text 

9.03 

133.36 

133.98 

41.23 

28 

Compressed  Speech 
with  Text 

8.02 

96.18 

94.05 

21.07 

22 

Total 

8.80* 

115.38** 

115.38 

42.43 

78 

*  TABK  reading  scores  ranged  from  3.1  to  12.9. 
'♦Individual  .Scores  ranged  from  45.00  to  268.00, 

TABLE  9 


Summary  of  ANCOVA 
for  Time  Scores  by  Group 
with  Reading  Scores  as  the  Covariate  Measure 


Source  of 
Variation 

df 

SS 

MS 

F 

Ratio 

Sign. 
ofF 

%of 
Var. 

TABE-Reading 

1 

442.42 

442.42 

.28 

.602 

0% 

Comprehension 

Treatment 

2 

19,077.62 

9,538.81 

5.93* 

.004* 

14% 

Residual 

74 

119,120.43 

1,609.74 

86% 

Total 

77 

138,640.46 

1,800.53 

100% 

♦Significant  at  &<.G1. 


In  order  to  find  out  which  pair  of  groups  were  significantly  different,  post- 
hoc  Tukoy  pairwise  comparisons  were  made.  The  results  of  these  poirwisu 
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comparisons  are  presented  in  Table  10.  To  gain  a  complete  picture  of  time  savings, 
time  elements  for  treatment  groups  were  compared.  (See  Table  11.) 

TABLE  10 


Post-E::  Pair-Comparisons 
for  Time  Scores  by  Group 
with  Reading  Scores  as  the  Covariate  Measure 


Text 
Only 
t/prob. 


Normal  Speech 
with  Text 
t/prob. 


Normal  Speech 
with  Text 


-1.91/.06 


Compressed  Speech 
with  Text 


-1.66/.10 


3.42/.001* 


"Significant  at  ft<.001. 


TABLE  11 

Comparison  of  Time  Savings 
in  Minutes  by  Treatment  Group 


Treatment 
Group 


Total 
Time* 


Lesson 
Only** 


Rewind 
Activ.*** 


Additional 
Time**** 


Text  Only 

Normal  Speech 
with  Text 


113.54 
133.98 


66.07 


67.91 


21% 
42% 


Compressed  Speech 
with  Text 


94.05 


44.05 


50.00 


*  * 

*  *  * 

*  *  *  * 


Total  time  =  Adjusted  means  for  each  group. 

Lesson  only  +  The  exact  audio  tape  lengths. 

The  rewind/activities  +  Total  tir«e  -  Lesson  Only. 

Additional  Time  =  Total  time/Compressed  Speech  Total 

Time  (the  shortest  total  time). 


Discussion 

The  statistical  result ;  of  the  study  showed  systematically  designed  text  to  be 
a  highly  effective  form  of  learning.  Since  all  groups  reached  a  ceiling  effect  with 
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regard  to  the  mastery  of  the  objectives,  augmentation  made  no  difference  with 
regard  to  the  effectiveness  of  the  materials. 

Another  finding  of  the  study  was  that  the  overall  variance  in  the  amount  of 
time  used  for  the  lesson  was  reduced  by  the  use  of  compressed  speech.  The  toxt  only 
group  had  the  largest  variance.  This  might  be  expected  since  the  partk  parts  paced 
themselves  by  their  individual  reading  rates.  The  normal  speech  with  tex',  group 
had  less  variance  that  could  be  a  result  of  the  mechanical  pacing  of  the  tape,  but  the 
variance  was  still  quite  large. 

Based  on  cognitive  theory,  these  large  variances  could  be  due  to  some 
participants  of  the  text  only  and  normal  speech  groups  having  to  repeat  portions  of 
the  instruction  because  of  their  inability  to  maintain  attention  at  the  slower  rate  of 
information  flow.  The  compressed  speech  with  text  group  had  the  smallest 
variance.  (See  the  graph  in  Table  8.)  If  the  theory  is  correct,  this  compact  variance 
may  be  due  to  the  participants'  ability  to  maintain  their  attention  on  the  instruction 
using  compressed  speech  augmentation.  The  difference  in  variances  could, 
however,  simply  be  due  to  the  total  instructional  time.  Since  compressed  speech 
shortens  the  instructional  time,  there  could  be  less  fatigue  occurring  in  this 
treatment  group.  This  raises  a  question  for  future  research  regarding  the  actual 
cause  of  the  reduction  of  variance. 

The  post-hoc  analysis  examining  only  the  time  feature  of  the  study  showed 
considerable  instructional  time  savings.  The  time  data  included  time  to  rewind 
when  necessary  and  to  complete  activities  interspersed  throughout  the  lessons; 
therefore,  the  time  savings  would  not  compare  to  studies  that  evaluate  time  savings 
using  word  rates.  For  example,  262  wpm  (compressed  speech)  is  1.5  times  or  50% 
faster  than  175  wpm  (normal  speech)  (Foulke  &  Sticht,  1967;  LaBarbera  & 
MacLachlan,  1979;  Sticht,  1971.)  However,  the  percentages  found  in  this  study 
should  give  a  more  accurate  picture  of  the  time  savings  that  could  be  expected  when 
compressed  speech  tapes  are  used  with  a  full  complement  of  text  and  practice 
activities.  See  Table  11  for  a  comparison  of  time  savings. 

In  a  media  center,  or  in  a  classroom,  the  small  variation  between  students 
and  instructional  time  savings  could  be  a  distinct  advantage.  Activities  could  be 
planned  around  more  predictable  beginning  and  ending  times  for  all  students. 
Since  systematically  designed  text  alone  is  already  highly  effective  and  efficient 
for  most  students,  a  cost-benefit  analysis  should  be  used  to  determine  if  the 
decreased  variance  and  time  savings  is  worth  the  additional  expense  of  producing 
the  compressed  speech  tape. 

With  regard  to  the  choice  of  augmentation,  the  statistical  results  of  the  study 
did,  however,  indicate  support  for  the  hypothesis  that  with  systematically  designed 
instruction  the  text  materials  augmented  with  compressed  speech  audio  tapes  are 
more  efficient  (concepts  learned  per  minute)  than  text  augmented  with  normal 
speech  audio  tapes;  however,  they  are  equally  as  efficient  as  text  alone. 

It  is  not  enough  to  assume  that  because  the  overall  effectiveness  of 
compressed  speech  was  equivalent  to  normal  speech  and  the  word  rate  was  faster, 
the  result  is  greater  efficiency.  The  results  of  this  study  were  calculated  using  an 
individualized  efficiency  formula  to  examine  the  efficiency  for  each  individual 
participant.  In  addition,  this  study  was  designed  to  investigate  the  actual  time  used 
to  complete  the  instruction,  rather  than,  the  word  rate  or  tape  length  alone.  The 
result  was,  that  even  with  the  ability  to  rewind  the  tape  and  listen  to  the  instruction 
again,  the  text  augmented  with  comoressed  speech  is  still  more  efficient  than  the 
text  augmented  with  normal  speech  and  equal  to  the  text  alone.  If  augmentation 
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were  used  with  systematically  designed  text,  compressed  speech  would  be  the  better 
choice. 

As  in  many  studies  (Foulke,  1967;  Orr,  1968;  Sticht  &  Friedman,  1964; 
Woodcock  &  Glark,  1969),  the  hypothesis  regarding  the  effectiveness  of  compressed 
speech  did  not  show  any  significant  increase  over  the  other  treatment  It  dtoas, 
however,  demonstrate  that  systematically  designed  materials  can  be  successfully 
compressed  without  any  significant  loss  in  comprehension.  The  combination  of 
compressed  speech  and  terse  text  does  not  apparently  ovCTcompress  the  information 
flow. 

Successful  comprehension  in  combination  with  the  other  findings 
regarding  efficiency,  time  savings,  and  reduced  variability,  domcastrate  that  the 
use  of  compressed  speech  tapes  to  augmer  systematically  designed  instruction 
may  be  a  valuable  addition  to  the  instruct!  i  process  waea  a  cost-benefit  is 
determined. 

Limitations  of  the  Study 

A  limitation  of  this  stuu /  and  otnvr  studies  using  systematically  designed 
materials  is  creating  a  "ceiling  effed,"  whon  examining  comprehension.  In  the 
process  of  development,  Uie  instructional  materials  are  revised  until  they  are 
highly  effective.  The  variance  in  performance,  in  effect,  has  been  engineered  out 
of  the  materials,  and  one  might  ?xneci  little  variance  in  effectiveness  among 
treatment  groups.  However,  other  variables  of  interest,  such  as,  time  and 
efficiency  (as  defined  in  this  ucudy)  require  that  effectiveness  be  measured,  even  if 
it  is  a  constant. 

Suggestions  for  Furthei  Research 

A  que^  cn  was  rgtaed  in  this  research  as  to  whether  compressed  speech  with 
text  indeed  reduces  fcho  variability  of  the  amount  of  time  needed  to  complete  the 
instruction  c-f  does  she  length  of  time  spent  on  the  instruction  impact  on  the 
variability.  U  the  variability  caused  by  unfilled  cognitive  capacity  or  by  fatigue? 
By  uring  systematically  designed  materials  to  exaggerate  the  ceiling  effect, 
c  i  ipreliension  could  be  controlled.  In  this  way,  the  instruction  could  be 
tengther^d  so  that  instructional  time  could  be  allowed  to  vary,  thereby  measuring 
lime  to  mastery  as  the  dependent  variable. 

The  cognitive  theory  in  this  study  has  set  forth  an  idea  that  rauding  rate 
Carver,  1982)  and  cognitive  capacity  are  related.  To  fully  examine  this  idea, 
future  studies  could  employ  a  reaction  time  (RT)  method  (Johnson  &  Heinz,  1978)  to 
find  cut  how  much  cognitive  capacity  is  being  used  when  people  are  reading  or 
listening  at  the  rauding  rate.  The  study  could  be  designed  much  the  same  as  this 
one  using  the  text  only,  normal  speech  with  text  and  compressed  3peech  with  text 
group.  Added  would  be  a  reaction  time  task,  such  as,  a  light  that  would  randomly 
turn  on  during  the  instruction.  The  length  of  the  reaction  time  that  it  would  take  the 
participants  to  push  a  button  indicating  that  they  saw  the  light  should  indicate  which 
group  was  using  the  most  cognitive  capacity. 

As  the  technology  for  compressed  speech  improves  to  the  point  where  there  is 
no  distortion  for  higher  levels  of  compression,  another  important  study  will  be  to 
investigate  the  effect  of  practice  on  the  comprehension  of  faster  and  faster  levels  of 
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compressed  speech.  This  research  may  provide  clues  as  to  the  limits  on  the  rate  of 
flow  the  brain  can  accommodate. 

Further  research  needs  to  include  larger  scale  studies  using  classroom 
situation  since  most  instruction  is  currently  delivered  in  this  manner.  Most 
studies  of  this  type  have  used  adults  in  military  or  university  settings. 
Researchers  should  consider  using  younger  participants  in  subject  areas  that  have 
a  high  degree  of  difficulty,  such  as  science. 

This  study  showed  a  large  variance  in  the  instructional  time  for  the  text 
only  group.  It  appears  that  compressed  speech  may  reduce  that  variance  for  the  slow 
readers.  Also,  the  question  as  to  whether  compressed  speech  can  increase 
comprehension  for  slow  readers  is  of  interest.  A  study  is  needed  that  examines 
compressed  speech  augmentation  with  instructional  materials  that  the  population 
has  more  difficulty  in  comprehending.  In  addition  to  a  reading  level  score,  a 
measure  of  the  participants'  habitual  reading  rate  should  be  calculated  to  use  as  a 
covariate  measure.  The  correlation  between  participants'  reading  level  and 
reading  rate  should  be  explored.  Results  of  this  type  of  study  would  determine 
whether  it  would  be  worthwhile  to  augment  traditional  school  texts  that  are  written 
above  the  capability  of  the  low  level  reading  population. 

Another  study  might  examine  whether  using  compressed  speech  with  text 
may  help  slow  readers  learn  a  new  cognitive  strategy  for  reading.  A  base-line 
reading  rate  and  comprehension  level  should  be  tested  at  the  beginning  of  the 
study.  A  control  group  would  use  the  text  only  throughout  the  study.  The  treatment 
group  would  be  given  compressed  speech  with  text.  After  each  increment  of  the 
study  reading  rate  and  comprehension  would  be  tested  on  a  reading  only  task.  The 
study  may  be  able  to  show  if  a  new  cognitive  strategy  for  reading  can  be  learned 
and  how  many  practice  sessions  with  compressed  speech  could  effect  a  change. 
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Thirty  Teaching  Strategies  Used  by  Teachers 
of  At-Risk  Students 

In  1989-90  Phi  Delta  Kappa  conducted  a  national  survey  in 
approximately  100  communities  in  North  America  which  involved  100 
schools  at  each  of  the  elementary,  junior  high,  and  high  school 
levels.  The  data  collected  from  the  teacher  survey  was  analyzed  at 
Indiana  State  University  by  the  authors.  Approximately  9,259 
teachers (2, 078  elementary,  2,822  junior  high,  and  4,359  senior  high 
school)  reported  strategies  they  regularly  used  with  at-risk 
students.  In  addition  the  instrument  collected  information 
necessary  to  develop  a  profile  of  the  typical  teacher  described  as 
white,  female,  forty-one  years  of  age,  and  holding  a  bachelor's 
degree.  The  average  length  of  teaching  experience  reported  by 
teachers  was  sixteen  years,  with  six-and-a-half  years  at  their 
current  school. 

Data  Analysis 

Data  analysis  concerning  teachers'  responses  to  thirty 
teaching  strategies  revealed  whether  teachers  used  each  strategy 
and  how  effective  they  felt  each  strategy  was  with  at-risk 
students.  Teachers  responded  "yes''  or  "no"  to  whether  they  used  the 
30  strategies  listed  and  rated  the  effectiveness  of  each  strategy 
on  a  four-point  scale  from  "not  very  effective"  to  very  effective. 
The  effectiveness  column  reflects  the  combining  of  "very  effective" 
and  "effective"  categories. 

A  rank  ordering  based  on  teachers'  "yes"  responses  to  their 
use  of  each  strategy  is  provided  for  elementary  teachers  in 
Table  1,  junior  high  teachers  in  Table  2,  and  senior  high  school 
teachers  in  Table  3.  The  responses  for  each  school  level  were 
further  divided  into  four  groups  according  to  the  rankings  on  the 
basis  of  "yes"  responses  of  75  percent  or  more,  50  to  74  percent, 
25  to  49  percent,  and  24  percent  or  less.  As  can  be  seen  in  tables 
1,  2,  and  3  eight  strategies  at  the  elementary  level  received  a  75 
percent  or  higher  use  while  five  strategies  received  this  level  of 
response  in  the  junior  high  and  senior  high  schools. 

Tables  1,  2,  and  3 
About  here 


Particularly  noteworthy  is  that  two  response  categories  at  all 
three  school  levels  reflect  92  percent  or  above  use:  (a)  notify 
parents  and  (b)  confer  with  parents.  These  two  categories  ranked 
either  one  or  two  at  all  three  school  levels.  The  rated 
ef fectivenesss  of  these  two  categories,  however,  did  not  place  them 
in  the  top  ten  strategies  for  the  elementary  teachers,  and  in 
places  10  for  junior  high  and  9  for  senior  high.  Elementary 
teachers  may  have  responded  less  than  the  other  two  groups  of 
teachers  because  of  mandatory  parent  conferences  for  many 
elementary  schools. 
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The  top  three  strategies  in  terms  of  effectiveness  at  the 
elementary  school  level  were  individualizing  instruction (95 
percent),  special  teachers (92  percent),  and  more  time  on  basic 
skills(9i  percent).  The  strategies  most  effective  at  the  junior 
high  school  were  individualized  instruction (89  percent),  smaller 
classes (86  percent) ,  and  more  time  on  basic  skills (85  percent) .  At 
the  senior  high  school  level  individualized  instruction  90 
percent),  smaller  classes(86  percent),  and  vocational  courses(85 
percent)  were  the  top  strategies  in  effectiveness. 

Use  of  more  time  on  the  basic  skills  was  reported  at  the  three 
school  levels (elementary  91  percent,  junior  high  82  percent,  and 
senior  high  82  percent)  which  supports  the  results  of  the  interest 
qenerated  in  the  media  and  from  testing  which  helped  to  establish 
the  need  for  more  basic  skills  developement  and  performance  among 
school  children.  Citizens  generally  feel  that  without  improvement 
in  the  basic  skills  students  are  going  nowhere  in  the  academic 
arena  or  in  the  workforce.  Furthermore  being  a  productive  citizen 
is  tied  to  the  acquirement  of  the  basic  skills. 

In  addition  to  the  top  three  ranked  strategies  at  the 
elementary  level,  five  other  strategies  deserve  attention  such  as 
emphasized  thinking  skills,  and  individualized  instruction. 
Ranking  elementary  teachers'  judgment  of  the  top  ten  strategies  in 
effectiveness  provides  the  following  list: 

Individualized  instruction (94  percent) 

Special  teachers (92  percent) 

More  time  on  basic  skills (91  percent) 

Smaller  classes (89  percent) 

Emphasize  thinking  skills (89  percent) 

Special  education (88  percent) 

Special  study  skills (87  percent) 

Emphasize  coping  skills (87  percent) 

Teacher  aides (85  percent) 

Peer  tutoring (84  percent) 
At    the    junior    high    school,    in    addition   to   the    five  top 
strategies  used,   a  list  of  the  the  top  ten  strategies  based  on 
judged  effectiveness  is  as  follows: 

Individualized  instruction (89  percent) 

Smaller  classes (86  percent) 

More  time  on  basic  skills  (85  percent) 

Special  education (85  percent) 

Special  teachers (85  percent) 

Special  study  skills (83  percent) 

Emphasize  coping  skills (82  percent) 

Emphasize  thinking  skills (82  percent) 

Vocational  courses (80  percent) 

Confer  with  parents (79  percent) 
At    the    senior   high    school,    in    addition   to   the    five  top 
strategies  used,  a  list  of  the  top  ten  strategies  based  on  judged 
effectiveness  is: 

Individualized  instruction (90  percent) 

Smaller  classes (86  percent) 

Vocational  courses (86  percent) 
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More  time  on  basic  skills (85  percent) 

Special  education (82  percent) 

Emphasize  thinking  skills (82  percent) 

Special  study  skills (81  percento 

Special  teachers (81  percent) 

Confer  with  parents (81  percent) 

Emphasize  coping  skills  (81  percent) 
It  is  interesting,  in  regard  to  teachers  judgment  concerning 
the  effectiveness  of  strategies,  that  at  all  three  school  levels 
individualized  instruction  was  at  the  top.  Smaller  classes  was 
second  at  two  levels  and  third  at  another.  Several  strategies 
occur  on  the  top  ten  effective  lists  for  the  three  levels  such  as 
emphasize  thinking  skills,  emphasize  coping  skills,  and  special 
teachers.  Keep  in  mind  that  educators  have  the  option  to  use  one 
or  more  of  the  strategies  concurrently.  Therefore,  infomation  from 
teachers  concerning  the  clustering  of  these  strategies  is  important 
because  it  probably  suggests  that  they  are  using  several  of  these 
strategies  concurrently  on  a  regular  basis. 

In  general,  the  elementary  teachers'  percent  ratings  were 
hiqher  than  the  junior  or  senior  high  teachers.  This  was 
especially  true  in  the  ratings  involving  effectl^n^s^: 
Apparently,  although  not  supported  by  hard  data,  the  at-risk 
student  at  the  elementary  level  is  not  so  pronouncedly  different 
from  peers  within  the  school.  At  the  junior  and  senior  high  school 
the  at-risk  students  become  more  segregated  or  recognized  and  their 
problems  become  more  obvious  and  pronounced  which  creates  more 
frustration  for  the  teacher  and  ii>  turn  less  positive  responses 
from  teachers. 

If  the  ten  most  effective  categories  are  reported,  then  tne 
reader  might  also  appreciate  the  reporting  of  the  six  categories 
for  each  school  level  judged  least  effective.  The  elementary 
teachers  reported  the  following  six  least  effective  strategies: 

Computerized  instruction  (56  percent) 

Before  school  programs  (46  percent) 

Extra  homework  (30  percent) 

Restrict  from  sports  (26  percent) 

Eliminate  art  and  music  (9  percent) 

Say  "leave  at  age  16"  (5  percent) 
The  junior  high  teachers  reported  the  following  six  least 
effective  strategies: 

Before  school  programs  (48  percent) 

Restrict  from  sports  (42  percent) 

Retain  in  grade  (42  percent) 

Extra  homework  (21  percent) 

Say  "leave  at  aye  16"  (15  percent) 

Eliminate  art  and  music  (9  percent) 
Finally,  the  senior  high  teachers  reported  the  following  six 
least  effective  strategies: 

Computerized  instruction  (46  percent) 

Retain  in  grade  (46  percent) 

Restrict  from  sports  (41  percent) 
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Extra  homework  (27  percent) 

Say  "leave  at  age  16"  (19  percent) 

Eliminate  art  c.  "  music  (9  percent) 
To  the  credit  of  all  teachers,  the  top  and  the  bottom 
effectiveness  rankings  of  strategies  are  very  similar  in  nature, 
with  the  exception  of  some  unique  teaching  level  differences  such 
as  "vocational  courses."  "Vocational  courses"  rank  in  the  bottom 
six  of  the  elementary  strategies  and  the  top  ten  of  senior  high. 
It  could  be  argued,  however,  that  there  are  more  similarities  than 
differences  at  all  three  levels  of  school  regarding  perceived  use 
and  effectiveness  of  strategies. 

Educators  and  computer  salespersons  who  make  predictions 
regarding  computer  education  in  the  year  2000  might  focus  their 
attention  on  how  "computer  instruction"  faired  at  the  three  school 
levels.  Notice  that  the  computer  strategy  made  two  of  the  least 
effective  strategies  lists.  It  is  interesting  to  note  that  most 
professors  who  are  predictiong  the  impact  of  computers  in  the 
twenty-first  century  only  use  a  computer  as  a  word  processor  in 
their  office. 

Attention  should  be  given  to  any  strategies  that  have  low  use 
by  teachers  and  yet  judged  to  have  high  effectiveness.  Only  two 
such  strategies  met  these  conditions  and  those  were  "restrict  from 
sports"  and  "home  tutoring".  Home  tutoring  was  veiwed  as  effective 
but  wa.3  reported  as  low  utilization.  Many  factors  could  account 
for  the  disparity  in  reporting  use  and  effectiveness,  but  most 
likely  would  be  that  there  are  not  sufficient  funds  to  afford  more 
home  tutoring.  The  other  strategy  in  question,  "restrict  from 
sports,"  arose  only  in  the  senior  high  level  responses.  Forty  one 
percent  of  the  senior  high  teachers  felt  that  restricting  from 
sports  was  effective,  even  though  only  four  percent  reported  using 
the  strategy.  Sports  are  deemed  very  important  by  the 
administration  and  the  community  by  senior  high.  Since  the 
questionnaire  to  collect  the  data  from  teachers  used  a  closed 
response,  there  is  no  data  to  suggest  the  thinking  processes  that 
accompanied  the  responses  by  the  teachers. 

It  would  take  many  more  pages  to  cover  all  of  the  data  and  the 
subtle  nuances  that  accompany  the  thirty  str  itegies  listed  in  this 
questionnaire.  While  the  list  does  not  exhaust  all  posssible 
strategies,  it  does  offer  some  of  the  more  common  and  more  popular 
strategies  now  being  employed  oy  teachers.  The  reader  can  return 
to  the  tables  and  surely  glean  even  more  information  of  interest, 
and  c^t  the  same  time  think  of  other  strategies  that  should  have 
been  included.  And  ultimately,  from  this  list,  future  researchers 
can  add  to  or  amend  the  strategies  for  more  in-depth  study  of  the 
concepts. 
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Table  1 


Elementary  School  Teachers  Responses  to  Effectiveness 
Ranked  by  Use  With  Reported  Effectiveness 


Percent                                            Strategies  Peycept 

Use                                        75  percent  or  more  Effective 

97.7  Confer  with  parents  83.0 
97 t 6  Notify  parents  82.4 
91,6  More  time  on  basic  skills  91.0 
90.2  Emphasize  thinking  skills  88.6 
89.2  Individualize  instruction  94.4 
87.2  Special  teacher  92.0 

78.8  Special  education  88.3 
75.2  Emphasize  coping  skills  86.8 


50-74  percent 

74.7  Flexible  scheduling  83.4 

73.1  Special  study  skilla  87.1 
70.5  Place  in  low  group  66.4 

69.2  Peer  tutoring  83.7 

68.8  Chapter  I  programs  83.1 

67.9  Refer  to  psychologist  7  5.5 
60.5  Teacher  aids  85.0 

56.7  Refer  to  social  worker  70.7 

56.5  Smaller  classes  89.4 

56.3  Summer  school  programs  73.7 

25-49  percent 

49.8  Special  textboks  72.3 

46.6  Retain  in  grade  59.6 
34.8  Computerized  classes  56.5 
28.8  After  school  programs  59.4 

26.4  Home  tutoring  68.9 

25.6  Extra  homework  30.3 

Less  than  25  person 

21.3  Alternative  school  65.3 

18.8  Vocational  courses  66.5 

16.7  Before  school  programs  46.0 
14.3  Restrict  from  sports  26.8 

5.2  Eliminate  art  and  music  9.5 

3.3  Say  "leave  at  age  16"  5.7 
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Table  2 

Junior  High  School  Teachers  Response  to  Effectiveness 
Ranked  by  Use  With  Reported  Effectiveness 


Percent  Strategies  Perpept 

'See  7  5  percent  or  more  Effective 


95.6 
95.5 
83.9 
82.0 
77.1 


69.5 
68.6 
64.9 
60.0 
63.8 
57.1 


47.9 
47.8 
47.6 
47.1 
45.8 
44.5 
44.0 
43.4 
39.8 
39.1 
25.1 


Notify  parents  1  *  I 

Confer  with  parents  79.0 

Emphasize  thinking  skills  81.6 

More  time  on  basic  skills  85.3 

Special  education  85 • 3 


50-74  percent 


74 . e                                   Individualize  instruction  89 . 1 

Special  study  skills  83*4 

Special  teachers  85*fJ 

Emphasize  coping  skil^  82.2 
Peer  tutoring 

Refer  to  poychologist  71.9 

^ ,  .  A                                    Refer  to  social  worker  71.1 

56 1 l                                    Summer  school  programs  66.5 

55 #o                                     Place  in  low  group  55.5 

25-49  present 


Chapter  I  pro-am  66  •  6 

After  school  programs  65.0 

Smaller  classes  35.5 

Special  books  71.0 

Vocational  courses  79.7 

Teacher  aids  77.4 

Retain  in  grade  42.1 

Flexible  scheduling  68.7 

Alternative  schools  70.3 

Restrict  from  sports  42.3 

Before  school  programs  48.3 

Less  than  25  percent 


21.8  ^owe  tutoring  60 

21.2  computerized  instruction  49.6 
18.6  Extra  homework  21.0 

10.3  Say  leave  at  16"  15.3 
"5.5  Eliminate  art  and  music  9.4 
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Table  3 

Senior  High  School  Teachers  Responses  to  Ef fecti/enesa 
Ranked  by  Use  With  Reported  Effectiveness 


Percent                                            Strategies  Percent 
Use                                         75  percent  or  more  Effective 

93.9  Notify  parents  78.7 

92.3  Confer  with  parents  81.3 
84.9  Emphasize  thinking  skills  81.6 
82.2  More  time  on  basic  skills  85.3 

76.8  Individualize  instruction  90.0 

50  -  74  percent 

67.7  Special  education  82.1 

66.6  Vocational  courses  85.7 
66.2  Special  study  skills  81.4 

66.0  Emphasize  coping  skills  81.1 

62.1  Peer  tutoring  80.5 

55.9  Summer  school  programs  71.7 

55.8  Special  teachers  81.3 
54.5  Refer  to  psychologist  68.6 

51.4  Refer  to  social  worker  69.0 

25-49  percent 

48.2  Special  textbooks  70.4 

47.7  Place  in  low  group  50.1 

46.3  Smaller  classes  86.4 
44.1  After  school  programs  62.4 

43.5  Teacher  aids  73.8 

43.4  Alternative  school  70.2 
43.3  Retain  in  grade  46.9 

39.9  Chapter  T  program  59.2 
39.0  Flexible  scheduling  62.1 

25.5  Before  school  programs  47.1 

Less  than  25  percent 

24.8  Home  tutoring  61.1 

18.9  Computerized  instruction  46.8 

24.0  Extra  homework  27.5 

13.1  Say  "leave  at  age  16"  19.7 
6.5  Eliminate  art  and  music  9.3 
4.7  Restrict  from  sports  41*5 
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Viewing  Hypermedia  Through  the  Prism  of  Evaluation 

CandkeW.  Gleim 
Francis  A.  Harvey 
Lehigh  University 


Introduction 

Hyp.  ext.  a  vision  conceived  in  the  interest  of  information  management  (Bush,  1945), 
has  had  an  initial  effect  on  the  "look  and  feel"  of  computer  interfaces,  creating  a  fusion  of  once 
isolated  dp  ications  and  information  sources.  Hypertext  (or  more  generally  hypermedia  when 
encompassing  video,  sound,  and  graphics  as  well  as  text)  not  only  supports  the  organization  and 
manag"«ent  of  information,  but  also  facilitates  the  presentation  and  exploration  of  informatton.  It 
is  the  ,tttr  capability  that  has  stimulated  considerable  debate  among  educators  concerned  with  the 
t!,  „v  v  and  practice  of  technology-based  instruction.  The  user-centered  experiential  nature  of 
hv,/,rmedia  technology  makes  it  a  likely  vehicle  for  delivery  of  instruction  (Allison  &  Hammond, 
( 7/0).  Unfortunately,  the  development  of  hypennedia-based  instructional  delivery  systems  is 
» jmoered  by  a  scarcity  of  research  which  can  be  used  to  guide  practice  and  with  problems 
associated  with  the  open-ended  design.  %     ,  . 

Evaluation  of  technology-based  instructional  programs  often  provides  insightful 
in  vestigation  of  an  innovation  concerning  salient  features  of  the  technology  and  how  the  features 
interact  in  an  applied  setting.  However,  evaluation  research  is  inherently  problematic  due  to 
competing  interests  of  organizations  and  individuals  involved  in  the  process;  these  problems  are 
not  (hVussed  in  this  paper  (see  Rossi  &  Freeman,  1989;  Reeves,  1991).  Instead,  the  purpose  of 
this  pa^er  is  twofold:  we  seek  to  establish  evaluation  as  an  important  concern  in  hypermedia 
research  and  development,  and  to  present  a  review  of  current  hypermedia  evaluation  research  with  a 
prescriptive  focus.  Characteristics  of  hypermedia  currently  attracting  the  attention  of  instructional 
developers  will  be  discussed.  Unfortunately  when  the  attributes  of  hypermedia  are  brought  into 
focus  by  recent  evaluation  research,  findings  are  often  limited;  conclusions  drawn  from  these 
developments  will  be  presented,  along  with  distinguishing  features  that  separate  the  evaluation 
approaches  employed.  Characteristics  of  hypermedia  which  present  challenges  to  evaluative  studies 
will  be  analyzed  and  possible  methodological  solutions  will  be  raised  Finally,  we  will  describe 
current  and  proposed  activities  at  Lehigh  which  seek  to  apply  the  ideas  presented  in  this  paper  to 
the  development  of  hypermedia  applications  for  technology  transfer  and  training  in  engineering  and 
other  areas. 

Hypermedia  and  Computer-Assisted  Learning 

The  interface  can  be  thought  of  as  the  first  gateway  to  effective  instructional  delivery  in  a 
computer-assisted  learning  (CAL)  system.  In  contrast  to  earlier  behavioristic  models  which 
required  the  learner  to  respond  in  a  specific  manner  and  measured  observed  behavior  (Skinner, 
1986),  more  recent  delivery  systems  influenced  by  a  cognitive  approach  attempt  to  provide  learning 
experiences  for  the  learner  so  that  the  learner  may  elaborate,  modify  and  integrate  the  material 
presented  with  previous  knowledge.  The  experiential  nature  of  hypermedia  meshes  with  the 
exploratory  and  generative  learning  orientation  of  cognitive  theory.  Consequently  hypermedia  is 
becoming  an  attractive  choice  for  some  CAL  developers  (Allison  &  Hammond,  1990).  However, 
the  open  environment  of  hypermedia  presents  new  challenges  in  designing  and  evaluating 
applications  specifically  to  achieve  learning  goals. 

The  principal  problem  related  to  using  hypermedia  as  a  delivery  mechanism  for 
instruction,  as  identified  by  Hammond  (1989),  concerns  the  learner's  experience  exploring  the 
information.  Exposure  to  large  bodies  of  unfamiliar  information  may  not  be  suitable  for  all  tasks 
or  all  individuals  and  may  result  in  confusion  or  indecisiveness  on  the  part  of  the  learner. 
Developing  an  overview  of  the  material  and  the  way  it  is  organized  can  be  difficult  for  the  learner; 
sometimes  merely  locating  information  that  the  individual  knows  is  included  in  the  application  can 
be  a  problem.  Connecting  the  information  in  a  myriad  of  ways  may  create  uncertainty  for  learners, 
who  may  become  hesitant  to  make  a  selection  because  they  are  unsure  of  what  to  e;;pect  or  of  what 
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is  expected  of  them.  Cognitive  resources  expended  in  attempting  to  master  the  hypermedia 
interface  may  interfere  with  the  intended  teaming  task. 

Chararteristics  of  HypErmedia  Annlicarinns  Designed  ffT  I  fflmW   

CurrenUy  there  are  no  universal  standards  governing  what  hypermedia  (or  hypertext) 
entails,  although  efforts  in  this  direction  are  being  formulated  (Moline,  Benigni,  &  Baronas, 
1990)  The  instructional  developer  must  depend  on  experience  and  intuition  to  design  and  develop 
an  application  incorporating  features  of  hypermedia  or  hypertext  Therefore,  characteristics  of 
hypermedia  for  learning  as  identified  in  literature  on  the  use  of  hypermedia  in  learning 
environments  will  be  explored  as  potential  variables  of  evaluation  studies. 

Gagne  and  Briggs  (1979)  identify  four  classes  of  variables  that  have  the  potential  to 
influence  an  evaluation  study:  process  variables,  outcome  variables,  support  variables,  and  aptitude 
variables.  Process  variables  refer  to  factors  in  an  instructional  situation  which  have  the  potential 
to  influence  outcome.  Outcome  variables  refer  to  measured  variables;  the  focus  of  the  study. 
Support  variables  are  represented  as  "opportunities  for  teaming"  (p.  297);  for  example,  the  presence 
of  adequate  materials  and  climate  represent  such  variables.  Aptitude  variables  represent  the  learner's 
ability  in  an  educational  situation. 

Process  variables  include:  instructional  sequence,  pacing,  and  events  of  instruction 
(Gagite*  &  Briggs,  1979).  Romiszowski  (1990)  suggests  the  focus  of  hypermedia  literature 
emphasizes  the  benefits  of  hypermedia  for  learning  (e.g.  non-linear  access,  learner-controlled, 
distributed  organization)  resulting  in  a  "process  oriented  philosophy"  (p.  322).  Viewing 
hypermedia  in  a  process  oriented  context,  as  characterized  in  the  literature,  separates  each 
characteristic  from  the  expected  outcome  or  goal  anticipated  by  using  hypermedia  for  teaming. 

The  products  of  learning,  or  outcome  variables,  are  often  characterized  in  cognitive  terms 
in  the  literature  describing  the  benefits  of  hypermedia.  For  example,  Kozma  (1991)  suggests  that 
knowledge  representation  may  exist  along  multiple  instead  of  single  conceptual  dimensions.  He 
also  suggests  that  complex  situations  generate  an  assembly  of  representations  instead  of  retrieving 
an  intact  schema;  hypermedia  may  facilitate  this  representation.  However,  traditional  tests  are  not 
designed  to  provide  such  cognitive  assessments. 

Attributes  which  emphasize  the  benefit  of  using  hypermedia  for  teaming  as  characterized 
in  literature  are  identified  in  Table  1  as  primary  process  characteristics.  Each  primary  process 
characteristic  (i.e.,  independent  of  other  characteristics)  is  presented  with  an  associated  product 
characteristic.  Secondary  characteristics  are  not  identified  in  this  table  and  include  those  factors  that 
are  influenced  by  and  have  the  potential  to  influence  additional  characteristics.  An  example  of  a 
secondary  characteristic  is  self-regulation;  Kinzie  and  Berdel  (1990)  suggest  self-regulation  is 
facilitated  by  learner  control.  To  elucidate  this  distinction,  an  example  representing  the 
relationship  between  learner-control  and  self-regulation  as  discussed  in  Kinzie  and  Berdel  is 
represented  in  Figure  2.  Although  Kinzie  and  Berdel  suggest  hypermedia  may  facilitate  self- 
regulation,  it  is  mediated  through  learner  control  and  therefore  does  not  appear  as  a  characteristic  in 
Table  1.  A  perspective  of  these  characteristics  is  available  through  the  filter  evaluation  provides. 

Recent  Developments  in  Evaluation  Research  on  Hypermedia 

Hypermedia,  as  it  pertains  to  instructional  systems  technology,  incorporates  many  aspects 
associated  with  computer-based  instruction  (CBI).  Research  conducted  on  factors  such  as  color, 
screen  arrangement,  video,  and  learner  control,  are  applicable  to  hypermedia  design.  Evaluation  of 
hypermedia  applications  becomes  more  problematic  than  directed  CBI  because  the  major 
characteristic  of  hypermedia,  the  ability  for  the  learner  to  traverse  the  application  in  a  multitude  of 
ways,  makes  each  learner's  experience  unique.  Browsing  and  navigation,  inherent  in  hypermedia, 
decrease  the  ability  to  control  the  experimental  condition. 

Our  objective  is  not  to  present  a  review  of  all  hypermedia  and  hypermedia-related 
evaluative  research.  Instead,  we  wish  to  concentrate  on  studies  of  hypermedia  evaluative  research 
used  in  educational  settings.  A  second  criteri    educational  applications  as  opposed  to  training 
applications,  further  narrowed  the  literature  selection  process.  Table  2,  and  2a  summarize  the 
foUowing  information  for  each  of  the  eleven  articles  (consisting  of  15  studies)  idenUfied  by  the 
criteria:  evaluation  approach  used,  focus  of  the  study,  indication  pertaining  to  support  of  research 
hypothesis,  instructional  content,  subjects,  and  an  indication  representing  the  number  of 
treatments. 
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Conclusions  Drawn  fmm  Evaluation  Research  on  Hypermedia 

Results  from  hypermedia  evaluation  studies,  identified  by  selection  criteria,  are 
consolidated  into  five  conclusions.  Each  conclusion  is  accompanied  with  an  associated  cautionary 
note.  Conclusions  and  cautionary  notes  are  discussed  below. 

Conclusion  1.  Hypermedia  study  guides  can  be  successfully  used  as  a  supplement  to  and 
substitute  far  lecture  instruction  (Higgins  A  Boone,  1990;  Horton,  Boone,  &  Lovitt,  1990).  The 
consistent  delivery  inherent  in  computerized  instruction,  offers  remediation  for  students  needing 
additional  instruction  to  master  subject  content,  or  assistance  to  students  that  have  missed  a  lesson. 
Learner-initiated  control  of  content  through  a  hypermedia  interface  provides  self-pacing,  which  may 
be  particularly  helpful  for  learning  disabled  students.  A  range  of  instructional  cues  (in  the  form  of 
directed  assistance)  depending  on  responses  to  multiple-choice  questions,  match  instructional 
remediation  level  with  learner's  highest  level  of  independent  comprehension. 

Caution  1.  Objective  outcome  measurements  used  in  comparative  studies  (Higgins  & 
Boone,  1990),  require  instructional  overlap  between  the  computer-assisted  content  and 
supplementary  lesson  materials  such  as  textbook  or  teacher  initiated  lecture.  Scarcity  of 
commercially  available  software  or  resources  necessary  for  development  of  quality  instruction 
which  match  specific  content  needs  of  the  instructor  are  prohibitive  factors.  Measured  gain  for 
each  learner  may  differ  depending  on:  learner  educational  ability,  relationships  between  various 
screens,  and  granularity  of  the  nodes  in  the  hypermedia  instruction  (cf.  Lanza  &  Roselli,  1991; 
Tripp  &  Roby,  1990). 

Conclusion  2.  Learning  performance  skills  (e.g.  problem  solving,  management 
practices,  group  interaction)  which  require  a  vast  range  of  experiences  can  be  mediated  through 
episodes  of  peer  modeling  using  video  illustrations  (Goldman  &  Barron,  1990;.  Lesson  planning 
time  for  the  instructor  is  easier  when  supplementary  materials  exist  on  videodisc. 

Caution  2.  Development  and  equipment  costs  for  hypermedia  may  far  exceed  costs  of 
traditional  instructional  methods.  A  "bottleneck,"  from  new  instructional  remedy  to  technological- 
based  teaching  tool,  may  discourage  experimenting  with  new  solutions  to  problems,  Although 
student  performance  skills  increased,  students  were  apprehensive  about  preparing  for  traditional 
examinations  after  hypermedia  instruction  was  substituted  for  the  lecture  (Goldman  &  Barron, 
1990).  Students  may  fe°J  more  confident  about  the  importance  of  information  when  it  is 
traditionally  delivered 

Conclusion  3.  Evaluation  of  hypermedia  enhancements  during  the  development  cycle  has 
the  potential  to  identify  deficiencies  of  the  product,  or  foster  new  ideas  to  improve  learner  strategies 
during  the  development  stage  (Egan  et  al.,  1989;  Covey,  1990).  Computer-based  monitoring  of 
the  frequency  with  which  the  learner  uses  available  strategies  and  of  learner  task  success  rates 
provide  a  rich  source  of  information  to  guide  subsequent  development.  Affective  measures 
obtained  early  in  the  development  process  may  eliminate  cosdy  ineffective  options  (e.g.  such  as 
digital  audio  feedback,  touch  screens,  etc.)  based  on  learner  reactions  to  pilot  versions* 

Caution  3.  Altering  more  than  one  function  in  an  instructional  application  between 
formative  evaluations  may  confound  the  ability  to  identify  the  source  of  improvement.  Studies  of 
this  nature  require  a  homogeneous  pool  of  subjects.  Although  subjects  may  be  comparable  with 
respect  to  many  demographic  variables,  groups  of  subjects  may  differ  in  aspects  not  compared  (e.g. 
computer  experience,  motivation,  anxiety,  history). 

Conclusion  4.  Metaphors  provide  a  meaningful  way  to  integrate  a  wide  range  of 
information  connected  by  theme  (Gay,  Trumbull,  &  Mazur,  1991).  The  learner  is  able  to  utilize 
the  information  in  an  intuitive  way. 

Caution  4.  Post  treatment  interviews  can  provide  insightful  affective  indicators 
pertaining  to  techniques  used  in  the  application,  but  matching  other  characteristics  of  the  learner 
with  comments  that  arise  during  the  interview  (e.g.  prior  knowledge  of  the  subject  matter,  ability 
to  direct  learning,  and  problem  solving  skills)  makes  generalization  difficult.  The  use  of  metaphor 
has  not  been  successful  in  all  systems  (Tripp  &  Roby,  1990).  When  combining  metaphors  and 
graphical  browsers  (or  using  browsers  alone,  Mays,  Kibby,  &  Watson,  1988),  elements  may 
compete  for  learners1  attention. 

Conclusion  5.  Relating  a  series  of  phenomena  encourages  interrelationa!  thinking 
(Landow,  1989).  As  demonstrated  through  class  discussions  and  examinations,  students 
contributed  a  greater  range  of  information  than  previous  classes.  Examination  responses  were 
judged  to  be  more  intellectually  sophisticated  and  more  rigorous  than  responses  from  previous 
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classes.  Students  appeared  more  motivated  to  contribute  their  ideas  of  inter-connectivity,  and  were 
perhaps  more  apt  to  complete  assigned  readings  because  they  felt  all  assignments  were  interrelafxL 

Caution  5.  Longitudinal  studies  may  indicate  novelty  or  duration  limitations. 
Implementing  a  study  which  evaluates  performance  over  a  period  of  treatments  may  provide 
evidence  that  performance  using  hypermedia  increases  over  time  (Higgins  &  Boone,  1990).  Direct 
transfer  of  hypermedia  experience,  to  course  content  may  occur  more  frequently  when  content 
included  in  hypermedia  instruction  is  contributed  based  on  the  recommendations  of  die  instructor. 
Conceptual  arrangement  of  the  hypermedia,  selected  by  the  instructor  or  designer,  may  have  an 
effect  on  learner's  ability  to  make  connections.  Giving  learners  the  ability  to  create  their  own 
links  may  influence  positive  reaction  to  hypermedia  As  Caution  1  mentioned,  developmental 
costs  and  instructors  expertise  may  limit  direct  transfer  of  any  computer-based  instruction  to 
course  content. 

These  conclusions,  as  shown  in  Table  2,  Table  2a,  encompass  different  approaches.  The 
next  section  compares  different  evaluation  approaches  used  by  researchers  as  outlined  in  the  tables. 

Evaluation  Research  Methodologies  Characterized  and  Discussed 

Several  studies  did  not  utilize  an  objective  outcome  variable  to  measure  learning  (Landow, 
1989;  Gay,  Trumbull,  &  Mazur,  1991;  Egan  et  al.,  1989;  Covey,  1990).  Instead,  the  study 
focused  on  outcome  which  measured  the  success  of  searching  through  a  hypermedia  application 
(Gay,  TVumbull,  &  Mazur,  1991;  Egan  et  al.,  1989),  or  used  an  essay  exam  which  was  judged 
with  interrater  reliability  measures  (Landow,  1989;  Egan  et  al.,  1989).  With  such  a  rich  open- 
ended  learning  environment,  what  approach  should  be  used  to  measure  the  impact  of  the  experience 
on  the  learner? 

The  methodological  question  is  often  answered  by  inquirer,  stakeholder,  board  members, 
community  members,  or  the  editor  responsible  forjudging  the  inquiry  (Guha  &  Lincoln,  1984). 
Often  professionals  will  select  an  approach  practiced  within  a  tradition  (cf.  Jacob,  1987;  Atkinson, 
Delamont,  &  Hammersley,  1988);  this  has  practical  value  as  researchers  within  a  professional 
community  can  exchange  results.  Each  approach  has  its  advantages,  but  two  general  approaches  as 
mentioned  by  Guba  and  Lincoln  (1984),  rationalistic1  and  naturalistic,  in  many  ways  can  be 
characterized  as  dichotomies  (see  Figure  1).  Although  they  are  often  mentioned  as  discrete 
categories,  they  may  be  more  appropriately  considered  as  ends  of  a  continuum. 

Regardless  of  the  form  of  inquiry,  Guba  and  Lincoln  (1984)  suggest  that  each  approach 
has  a  set  of  criteria  broadly  defined  into  four  categories  which  can  be  used  to  identify  the 
trustworthiness  of  the  study's  findings  (see  Table  3).  Truth  value  represents  the  confidence  of  the 
findings;  applicability  describes  suitability  of  findings  to  similar  inquiries  with  different 
setting/subject  arrangement;  consistency  refers  to  the  ability  to  replicate  the  study  ;  and  neutrality 
represents  the  degree  to  which  the  findings  are  established  without  biases. 

Classified  as  rationalistic  methods  by  Guba  and  Lincoln  (1984),  experimental  and 
quasi-experimental  inquires  have  a  long  tradition  modeled  after  scientific  forms  of  inquiry 
(Campbell  &  Stanley,  1963;  Cook  &  Campbell,  1979).  The  distinction  between  the  two  rests  on 
scheduling,  either  measurement  or  exposure.  Where  experimental  designs  are  more  tightly 
controlled,  experimental  rigor  is  not  abandon  in  situations  where,  for  example,  random  assignment 
to  treatment  is  impossible.  Instead  quasi-experimental  techniques  are  employed 

Formative  evaluation  was  used  in  a  few  of  the  evaluation  studies  reviewed.  Labeled 
a  proactive  approach  by  Stufflebean  (1984),  this  technique  offers  systematic  indication  of  the 
merits  of  specific  options  prior  to  completion  of  product  development,  in  contrast  to  summative 
design  which  evaluates  at  the  end  of  development 

Altltough  Stufflebean's  (1984)  and  Scrivcn's  models  go  beyond  mere  product  evaluation, 
Dick  and  Carey  (1990)  prescribe  formative  approaches  that  are  appropriate  for  instructional 
products.  The  formative  evaluation  draws  upon  the  systematic  practices  of  quasi-experimental  and 
experimental  design.  Yet,  in  practice,  it  takes  on  a  range  of  techniques  (cf.  Covey,  1990;  Egan  et 
al.,  1989).  This  technique  seeks  convincing  evidence  (Gagn6  &  Briggs,  1979)  and  hence  lacks 
correlates  for  all  categories  for  comparison  in  Table  3. 

Naturalistic  inquiries  use  several  techniques  to  guard  against  accusations  of  less  rigor 
than  rationalistic  studies.  For  example,  triangulation,  a  mechanism  to  cross- verify  data  with 
professionals  of  varying  perspectives,  is  analogous  to  objectivity  in  rationalistic  studies  (Guba  & 
Lincoln,  1984). 
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Interrelationship  of  Characteristics  of  Hypermedia  and  Results  from 

Evaluation  Studies 

.,  Jldow  am)  found,  through  naturalistic  inquiry,  an  improvement  in  the  quality  and 
deoth  of  to  c^ams.  Providing  the  learner  with  extension  and  cons^t^abduy 

Sua  W9?S 'taSe  the  learner's  capacity  to  interrelate  ideas.  Node  size  andrepr^sen.auon 
Si  by  lmTmay  also  contribute  to  success.  However.  cunenUy  there  is  no  evince  to 
s^thVnS  Z  experience  with  hypermedia  fosters  transfer  of  learners'  capacity  to 

i— aSSS  (1989),  hypermedia  does  facilitate  learners'  ability  to  control 
te«oJSX^$Wt^*  Berdel,  1990;  Hammond.  1989),  MMv 
Sinsa\S^oaw(1990)  fou..d  differences  in  the  rate  of  gain  between  groups  of  diffewntabdity. 
^SSSSSmS^^  interacting  effects  of:  organization  of  the  hvpemedm  applwauon 
^l^Ss^^andl^tiide  level  of  the  learner?  Is  this  control  best  fostered  when  it  is 
STtocr^Snt!  or Xn  it  is  used  to ;  con**  instructional  .trategKS  (e.g.  examples, 

exr^nc7wTS7drawupor  However,  they  have  failed  education  wim 
rXmri^  Zr^  anW^h^hat  is  often  used  to  investigate  the  effects  of  technology  and 
SKorm^of  Sleeves,  1991;Clark,  1985;  ClarM983). ^Idtmelited  to 
™nr«iches  10  desimi  research  studies  require  studies  to  be  designed,  planned,  and  implemented  to 
m^^S^^  (Kerimger,  1973).  Traditional  approaches  to  evaluation  may 
™T™viK^^  innovations  when  the  focus  of  the  study  concerns 

(e  e  linear  over  nonlinear  acquisition,  cf.  Lanza  &  Rozelli,  1991;  Mayes,  Kibby  &  Watson, 
Q^  TaSSS  approach  may  provide  effective  reseaich  designs  when  research 

uSon^^ 

StLtment  with  acontrol  measure  to  monitor  maturity  (e.g.  design  8,  Campbell  &  Stanley, 
SafcS wl*n  an ^inSon  of  comparative  effects  of  distinctively  different  raiments  is  desired 

(AW"MThe^^e9o1  quasi^xperimental  evaluation  rests  on  the  additional  attention  die 
e  valuator  SVlhe  threats  to  internal  validity  (without  randomking  subjects  to  study 
g^  iextrr^aTdity  (time  requirement  to  conduct  such  a  study),  and  the  statistical 

r^fnhy^  t 
objectives  S^l^nl^  to  contentK>riented  as  Bonner  (1988)  suggests?  Self-tests, 
cSS  «Lrs  or  analyzers,  may  aid  self-discovery  while  tracing  the  kWs  pry 
™vMiJ  develooers  with  mformation  that  may  assess  progress.  Gay  and  Raffensperger  (1989) 
™ntiro  aSc^  n^ures  giving  students  the  capability  to  self-test  their 

oroS  iSSXvideo  segment  with  an  instruction*!  overlay .  The  learner  is  mv.ted  to 
SSurSerSnJ  in  a  graphic*  environment,  by  manipulating  variables,  thus  demonstrating 
understantog^i^^  ^  ^  ^  m 

defined  (Winn  1990)  Winn  asserts  that  prescriptions  for  cognitive  instructional  sttategies  are 
fe^iZeteLLton  is  underway,  as  opposed  to  defining  them  early  in  the  ftevelopment 
c^aTinTtovIral  instructional  strategies.  The  teacher  or  designer  mustar^ror^^cthe 
nc^OT^eS  when  practicing  a  Cognitive  approach  to  instruction,  .ither  must  monitor 
^Sed  Ener  and  provide  mechanisms  to  change  strategies  when  appropriate.  This  also 
sumS afwmative  design  strategy  to  insuie  that  the  designer  can  monitor  actual  use  of  the 

Planned  interventions,  depending  on  the  way  they  are  implemented  (e.g.,  by  the  teacher 
instead  of  Z  arolication)  may  alteV  learners'  success  depending  on  attributes  present  at  the  tone  of 
Z^^lZ2ng  additive  effects  of  the  interaction  may  require  a  modelsuch  as  those 
urt uSE ra£  science  (SaScevic,  Mokros,  &  Su,  1990).  This  form  of  f^^j^ 
model  captures  and  measures  the  attributes  f  the ^  interaction  process  and  specific  attributes  of 
memt?p^c.^ngm  the  interaction.  This  naturalistic  approach .provides  an  analyses  of  what 
nBnspu^  during  interaction  witliout  loss  of  descriptive  power  which  typically  occurs  in  a 
statistical  study. 
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The  effect  of  hypermedia  on  learner  strategy  formulation,  although  posed  by.nany 
researchers  when  hypothesizing  the  value  of  hypermedia  as  an  instructw^  rr^urn  has  not  bee,, 
evaluated.  Similarly,  evaluation  of  levels  of  instructional  suppwt  is  needed.  Altiiough 
hypermedia  is  designed  to  inspire  the  learner  to  control  his/her  own  teammg,  in^uonal 
Sgies  designed  to  provide  limited  guidance,  yet  unending  support,  may  remedy  the  problems 
associated  with  hypermedia. 

Evaluation  of  Hypermedia  at  Lehigh  University 

Since  Set  jmber,  1989,  faculty  and  staff  of  the  Advanced  InfwmatKMi  Techitologies 
Laboratory  (AITL)  of  the  College  of  Education  at  Lehigh  University  have  been  engaged  in 
dwdfaptag  uitegrated  hypermedia  materials  for  a  number  of  clients.  The  current  major  projects  of 
Z^  S^Sii  of  Dr.  Francis  A.  Harvey,  are  the  Hypermedia  Support  System  for 
Z  Bridg;^ueInvestigator  (Hyper-BFl)  and  the  Cities  in  Schools  HypemiediaDevelop.Knt 
Project (CIS/HDP),  b.  this  section  of  the  paper  we  will  oescrlbed  current  and  planned  evaluation 
efforts  related  to  those  projects. 

Muaiiarj  nf  the  Hyjcopgdja  Suppafl  System  for  the  Bridge,  Fatigue  Investigator 

The  Hy^edia  Support  System  for  the  Bridge  Fatigue  Investigator 'Project  is  a 
subproject  of  Lehigh's  Center  for  Advanced  Technologies  for  Large  Structural  Systcms(ATLSS) 
soonsored  by  the  National  Science  Foundation.  The  project  is  part  of  a  program  which,  for  the 
SelSrs  nas  been  exploring  the  use  of  combinations  of  knowledgcbased  system  technology 
and  hypermedia  technology  for  information  utilization  and  dissemination,  training,  and  technology 
transfer  (see  Harvey,  1990;  Harvey,  1989).  . 

The  Bridge  Fatigue  Investigator  (BFI)  is  an  expert  system  designed  to  aid  bridge 
inspection  (Michalerya,  Wilson,  &  Chen,  1990).  An  interdisciplinary 'team  of  subject  matter 
experts  (including  the  developers  of  3FI)  and  instructional  designers  is  developing  a  hypermedia 
support  system  to  train  bridge  inspectors  in  the  use  of  BFI  and  to  help  professional  engineers  use 
BFI  for  post-inspection  bridge  analysis.   

The  Hyper-BFI  Project  is  now  in  its  beginning  stages.  Techniques  used  to  support 
formative  evaluation  include  using  rapid  prototyping  to  create  a  pre-prototype  for  team  members  to 
react  to  small  slices  of  system  development  during  team  meetings.  A  special  icon  represented  by 
an  exclamation  symbol  is  operational  from  any  instructional  screen.  This  is  used  to  communicate 
problems  with  the  screen  directives,  or  navigational  aids  at  any  tome  during  i course  of 
instruction  while  a  development  team  member  is  viewing  the  application  A^tionally ^selected- 
response  measure  with  Likert  scale  format  is  also  being  considered  to  gather  feedback  from  users 

during  P»°JJ|^|*  design  ^jg^es  at  mis  stage  concem  local  aspects  of  the  system:  the  design 
of  a  reliable  instrument  which  will  measure  learner  attributions  and  beliefs  with  resptct  to  the 
developing  system  (Clark,  19J3),  and  the  identification  of  specification  misconceptions  in  design 
implementetion.  These  evaluation  mechanisms,  where  possible,  will  be  incorporated  into  the  pre- 
prototype  hypermedia  application  as  it  is  developed. 

Evaluation  of  the  Cities  in  Schf*^  Hypermedia  Development  Project 

The  Cities  in  Schools  Hypermedia  Development  Project  has,  for  the  past  three  years, 
been  developing  an  integrated  hypermedia  training  curriculum  for  Cities  in  Schools,  Inc.  (CIS),  the 
nation  s  largest  and  most  successful  progiam  addressing  the  needs  of  at-risk  youth.  A  four-week 
fuU-day  integrated  hypermedia  training  course  for  directors  of  CIS  local  projects  has  been  developed 
which  includes  nearly  60  minutes  of  professionally  produced  motion  video  and  awwoximately  500 
still  photographs  on  videodisc;  over  500  pages  of  printed  instructional  nsojin^and  background 
material;  and  a  comprehensive  hypermedia  computer  program  with  several  hundred  computer-based 
instructional  resource  materials  and  navigational  aids  which  allow  both  trainer  and  trainees  to 
access,  use,  and  interrelate  computer-ba  jed,  video,  and  print  instructional  materials  (see  Harvey, 

1990;  Harvey  &  S^ry,  1989).  ,  . . 

A  formative  evaluation  of  the  Cities  in  Schools  Project  Operations  integrated  hypermedia 
training  course  was  carried  out  both  during  the  training  program  for  Cities  in  Schools  trainers 
conducted   l^high  in  February,  1991  and  during  the  initial  training  program  delivered  by  Cities 
in  Schools  trainers  in  March,  1991  (Spokane,  1991).  This  study  used  both  trained  observers  and 
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videotape  to  gather  a  comprehensive  picture  of  how  the  training  course  was  use^  h^  tramm 
IndSe^responded  to  it.  These  observations  were  siipptemented  by  open-ended  feedback  forms 
mmnletad  bv  trainers  and  trainees  both  during  and  after  training. 

"  P    SincJcS  ^Schools  training  programs  arc  held  monthly  in  the bfctional  Center  for 
Partnership  Development  hypermedia  training  laboratory  at  Lehigh,  ample  opportunity  k  provided 
u  conduct  both  formative  evaluation  of  CIS  hypermedia  materials  under  devetopment  and 
sun^e^luationof  those  materials  al^dy  m  use  AfoUow;upevalu^onof  the  Cues  m 
Schools  Project  Operatiotis  training  course  is  planned  for  March,  1992  at  Lenign. 

^eCidesin  Schools  integrated  hypermedia  training  course  for  executive  directors  of 
city-wide  CIS  programs  now  under  development  will  incorporate  in  ite  software  unob^sive 
nwhnisms  f£  recording  complete  intonation  about  how  individual  users  make  use  of  the 
ES^Stor^HarVey,  1990).  Each  interaction  with  the  software  (setecuon  of icon,  text 
X^f  navuitional  support  tools,  etc.)  wiU  be  recorded  and  time-coded  .automatically.  A 
^  rc^rd  of  me  mteraSbetween  user  and  the  hypermedia  program  will  be  recorded  on  disk 

and  *  (cr  win  ted  out  for  analysis*  . 

'    i„  both  of  these  projects,  the  evaluation  components  are  designed  at  the  same  tinw  as  the 
application  is  designed.  Design,  development,  and  evaluation  are  seen  as  parallel  and  mutually 
Stive  activities.  This  approach  may  ameliorate  the  "trapped  admuustrator;  syndrome 
MeS  by  Campbell  (1977),  whereby  the  administrator  is  faced  with  the  design  of  an  evaluation 
study  after  the  program  is  fully  developed  and  in  place, 

Conclusion 

Evaluation  often  distorts  our  view  of  instructional  programs  and  the  benefits  provided  oy 
new  technologies  (Reeves,  1991).  Consequendy,  developers  of  iiistructional  software  and 
educators  a!ike  use  public  receptivity  as  a  barometer  of  success  for  their  new  products. 
Unfortunately,  this  offers  minimal  support  in  acknowledging  the  product  as  providing  efficacy  in 
instructional  environments.  .......      .  „.  „<• 

The  benefit  associated  with  hypermedia  evaluation  research  hes  in  the  dissemination  ot 
relevant  findings  and  impact  of  the  find  ngs  to  enhance  future  practice.  Research  designs 
implemented  with  a  rationalistic  approach  need  valid  constructs  and  measures  to  accurately  quantify 
the  variables  related  to  the  focus  of  the  study.  Plausiole  rival  hypotheses  can  offer  rich 
descriptions  by  contradicting  a  study's  findings  as  a  natural  consequence  of  using  rationahsUc 

methods  ^2^^^.^  ^  ^  potential  to  contribute  rich  descriptions  needed  to  fos'er 
new  ideas  and  provide  insight,  Ouba  and  Lincoln  (1984)  offer  t*ee  tests  to detem»ne»fa 
naturalistic  appWh  should  be  selected  over  rationalistic  methods:  1)  data  from  study  to  study  can 
not  be  aggregated;  2)  researcher  has  little  control  of  extraneous  variance;  and  3)  impact  of  research 
on  practice  is  absent  By  these  standards  hypermedia,  and  many  other  focused  studies  of 
technology,  is  in  ideal  topic  for  naturalistic  evaluation.  ....     A  .... 

Despite  the  correlation  of  judgement  criteria  between  rauonalistic  and  naturalistic 
approaches  (Table  3),  the  procedures  arc  very  different  Naturalistic  study  requires  exceedingly  large 
quantities  of  data  as  the  approach  is  inductive  instead  of  deductive.  (In  fact  hypermedia  has  been 
used  to  track  and  to  interrelate  the  various  data  repositories  in  a  natwulistic  evaluation,  see  Martin, 
1991 )  Practitioners  must  become  aware  of  naturalistic  approaches  to  evaluation,  their  limitations 
and  potential  benefit  to  retrofit  development.  In  particular,  how  would  naturalistic  inquiries 
remedy  Cuba  and  Lira's  three-test  scenario?  Could  date  be  aggregated    comparative  analysis? 
Could  researcher  methods  be  analyzed?  !iow  would  naturalistic  studies  influence  practice? 

The  review  and  analysis  in  this  paper  suggests  that  hypermedia  requires  an  adaptive 
interactive  research  model  to  investigate  the  effects  of  treatment  It  is  probable  that  this  model 
will  incorporate  elements  of  rationalistic  and  naturalistic  approaches.  ReCardless  of  the  method 
used  to  evaluate  the  effectiveness  of  hypermedia,  a  focus  on  communicating  the  results  of  the  study 
will  foster  progress  in  hypermedia  development  for  learning. 
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Footnotes 


iGuba  and  Lincoln  (1984)  use  the  term  "rationalistic"  to  reference  forms  of  conventional 
inquiry  such  as  experimental  and  quasi-experimental  forms  of  evaluation. 


257  2S1 


Oleim  and  Harvey 


Hypermedia  Evaluation 


rationalistic 

I— 


naturalistic 


nomothetic 

quantitative 

theory  -based 

greater  precision, 
mathematically 
manipulate 

research  designs  are  develope 
to  test  theories 

perfection 


ideographic 

qualitative 

inquiry  -based 

descriptively  rich, 
accommodate 
non-numeric  phenomena 

theories  are  developed 
to  explain  data 

adaptability 


Figure  1:  Comparison  of  rationalistic  and  iiaturalistic  methodologies 


Figure  2:  Relationship  between  learner  control  (primary  characteristic)  and 
self-regulation  (secondary  characteristic). 


9 

ERIC 


258 


282 


Gleim  and  Harvey 


Hypermedia  Evaluation 
14 


Table  1 

rhnrnrtPrlsticyj  nf  hvpprmerfi*  (Pnwe&A  and  ftxnectcd  OlltfOmf  (Product) 


Cited  Literature 


Process  Characteristic 


Outcome  Characteristic 


Jones  (1990) 


Kozma(1991) 


Kozma(1991) 


Hypeimedia  more  accurately         Evidence  supporting  nonlinear 
reflects  the  way  the  mind  actually  over  linear  acquisition, 
works,  i.e.,  thru  the  association 
of  ideas  rather  than  in  a  linear  or 
hierarchical  fashion. 


Hypen  cdia  facilitates  application 
and  transfer  of  complex 
knowledge  to  new  situations  via 
multiple  exploration  on  either 
thematic  or  conceptual 
dimensions. 


Evidence  supporting  knowledge 
representation  along  multiple  * 
instead  of  singular  conceptual 
dimensions.  Assembly  of 
representations  instead  of  retrieval 
of  schema. 


Hypermedia  provides  learner  with  Evidence  supporting  Hypermedia 
extension  and  construction  prompts  learners  to  think  about 

ability.  ideas,  how  they  are  interrelated, 

structured 


Locatis,  Letourneau,  & 
Banvard(1989) 


I  jocatis,  Letourneau,  & 
Banvaid(1989) 


1/xatis,  Letourneau,  & 
Banvard  (1989);  Hammond 
(1989) 


Marchionini  (1988);  Kinzie  & 
Berdel  (1990);  Hammond 
(1989) 

Marchionini  (1988) 


Marchionini  (1988) 


Hypermedia  requires  linking  with 
specific  forms  of  instructional 
support. 


Hypeimedia  qpen~ended 
experience  may  be  different 
depending  on  previous  exposure 
to  content  domain  and  aptitude 
level. 


Hypermedia  offers  new  ways  of 
learning  how  to  learn. 

Potential  to  alter  roles  of  teachers 
and  learners  and  facilitate 
interaction  between  them. 


Evidence  which  suggests  success 
based  on  different  forms  of 
linking. 

Evidence  which  indicates  that 
different  arrangements  aid/impact 
success  rates. 

Evidence  which  indicates  novice, 
intermediate,  and  expert  skill 
require/use  different  treatment 


Evidence  which  indicates  this  is 
beneficial. 


Evidence  of  increase  in  strategies 
for  the  learner. 

Evidence  of  Human-Computer- 
Intermediary  support 


Hypermedia  node  size  and 
representation  may  affect  use. 


Hypermedia  facilitates  users1 
ability  to  control  their  own 
learning. 
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Table  2 

Selected  Evaluation  Research  Table  1  of  2 


Study 


Approach 


Study 
Supported 
Research 
Study  Focus  Hypothesis 


Content 


Subjects 


Multiple 

Measures 


Covey  (1990)  Formative 


Formative 


Egan  et  al. 
(1989) 


Egan  et  al. 
(1989) 
Evaluation 
Study  II. 


Egan  et  al. 
(1989) 
Evaluation 
Study  in. 


Gay,  Trumbull,  Quasi- 
Mazur(1991)  experimental 
(case  study 
using 

volunteers) 


Formative 
(between 
subjects 
measurement) 


Formative 


Goldman  & 
Barron  (1990) 


Naturalistic 


Hammond,  & 
Allinson  (1988) 

Higgins  & 
Boone  (1990) 


Exploring  no 
knowledge 
evidentiary 

issues: 

attribution  and 
critical  analysis 
in  study  of 
visual  art 

Print  search  no 
compared  to 
computer  search 


Added  open  yes 
ended  questions, 
affective  rating 
measures 


Smaller  screen  yes 
version 
compared  to 
print  search 


Fine  Art 


Formative 


Quasi- 
experimental 
(pretest-posttest 
&  retention, 
random 
assignment) 


Texas  Legal 
Qxte 


Texas  Legal 
Code 


Texas  Legal 
Code 


Interaction  of 
choice  of  search 
mode  and 
Learning  Study 
Skills 

Affective 

measures, 

traditional  test 

performance, 

teaching 

methods 

improvement 

Performance 


Comparing 
delivery  modes: 
Lecture, 
Computer 
Study  Guide, 
&both 


not  clear 


Cultural 
Entomology 


mixed 


Mathematics 

teaching 

methodology 


n/a 


no 


Cognitive 
Psychology 

Social  Studies 


College  under- 
graduates 


no 


College  under- 
graduates (with 
1  or  more 
statistics 
courses) 

College  under- 
graduates (with 
1  or  more 
statistics 
courses) 

College  under- 
graduates (with 
1  or  more 
statistics 
courses) 

College  under- 
graduates 


yes 


yes 


yes 


no 


Preservice 
college  students 


College  under- 
gixhiates 

High  school 

students: 

regular, 

learning 

disabled,  & 

remedial. 


no 


yes 
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Table  2a 

SfilfifilBd  Evaluation  Research  Table  2  of  2 


Study 

HJggins  & 

Bocvte  (199(9 
Such*  II 


Approach 


Study  Focus 


Sfcdy 
Supported 
Research 
Hypothesis 


Quasi-  Increase  in  test 

experimental  performance 
(follow-up  A-B-  using  study 


yes 


A) 


Horton,  Boone,  Quasi- 
Lovitt  ( 1990)  experimental 
(small  case) 


Landow  (1989)  Naturalistic, 
ethnographic 


Lanza  &  Quasi- 
Roselli  (1991)  experimental 

(post-test  case 

study) 


Mays,  Kibby, 
&  Watson 
(1988) 


Mays,  Kibby, 
&  Watson 
(1988) 
Study  II 

Tripp  &  Roby 
(1990) 


Experimental 


Naturalistic 


Experimental 
(post-test) 


yes 


yes 


guide  as  follow- 
up  instruction. 

Increase  in 

comprehension 

using 

instructional 
sequencing^ 
study  guide 

Increase 
studer^s 
anaLucal 
skills,  and 
develop  ability 
to  assimilate 
breadth  of 
information 


Effectiveness  of  not  clear 

hypertext 

instruction  vs 

CAI 

Comparison  of  no 
hypermedia  and 
traditional 
teaching 


Protocol 
Analysis 


Visual 
metaphors 
increase 
learning,  rote 
learning  is  not 
more  effective 
with 

hypermedia 


not  clear 


no 


Content 


Subjects 


Social  Studies 


Washington 
S  tate  History 


Survey  of 
English 

Literature,  1700 
to  the  Present 


Learning 
disabled,  & 
remedial 


Learning 
disabled  high 
school 
students 


College  under- 
graduates 


Pascal 

programming 


Geology 


Geology 


Multiple 
Measures 


yes 


yes 


n/a 


College  under-  no 
graduates 


College  under-  no 
graduates 


College  under- 
graduates 


Japaneese-  College 
English  lexicon  graduates,  and 
under-graduates 


no 


n/a -not  applicable 
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fritfjia  far  Judging  tfrc  Trustworthiness  of  the  Findings  of  an  Evaluation  Study 


Approach 


Criteria  for 

Judgement  Experimental 


Quasi-Experimental    Formative  Naturalistic 


Convincing  Credibility 

Transferability 
Dependability 
Confirmability 


Truth  Value  Internal  Validly  Internal  Validity 

Applicability  External  Validity  External  Validity 

Consistency  Reliability  Reliability 

Neutrality  Objectivity  Objectivity 


Mote.  The  data  in  column  1,2  and  5  are  from  "Epistemological  and  Methodological  Bases  of 
Naturalistic  Inquiry"  by  E.  0.  Guba  and  Y.S.  Lincoln  ,  Evaluation  Models  (p.  326)  in  G.F. 
Madaus,  M.  Scriven,  and  D.L.  Stufflebeam,  (Eds.),  1983,  Boston:  Kluwer-Nijhoff  Publishing. 
Adapted  by  the  authors. 
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Background 

The  professional  isolation  of  educators  has  long  been  a  problem.  Dan  Lortie 
describes  the  "egg  crate"  nature  of  teaching,  referring  to  the  fact  that  typically,  teachers 
are  isolated  in  their  classrooms  and  have  little  opportunity  for  interaction  with  other 
teachers  and  experts  in  their  fields  (Lortie.  1975).  Supporting  Lortie's  findings.  Goodlad 
(1983)  reports  that  teachers  are  generally  confined  to  their  classrooms  and  are  unable 
to  converse  with  other  educators  about  problems  or  concerns. 

The  capabilities  provided  by  electronic  communication  networks  provide  a 
possible  solution  for  the  professional  isolation  of  teachers.  Although  research  suggest? 
electronic  communication  networks  are  currently  being  used  effectively  by  some 
educators.the  diffusion  process  of  such  networks  has  not  been  widely  spread.  With  the 
development  of  electronic  communication  networks,  an  adoption/diffusion  cycle 
occurs  as  potent'al  users  become  aware  of  the  innovation,  judge  its  relative  value,  make 
a  decision  based  on  that  Judgement,  implement  or  reject  the  innovation,  and  seek 
confirmation  of  the  adoption/rejection  decision.    Electronic  communication  networks 
are  one  of  the  most  recent  technologies  to  begin  the  diffusion  process  through  the 
educational  system.  Research  on  electronic  communication  networks  is  valuable  to 
educators  developing  and  managing  networks.  More  specifically,  research  determining 
characteristics  of  early  adopters  and  how  they  Judge  the  value  of  such  systems  is 
necessary  because  this  information  helps  researchers  and  developers  understand  and 
improve  use  of  the  innovation.  The  adoption/diffusion  theory  (Rogers.  1986)  provides  a 
framework  in  which  this  research  can  be  conducted. 

Rogers  (1986)  stated  that  the  four  main  elements  in  the  adoption/ diffusion 
process  are  the  innovation,  communication  channels,  time  and  the  social  system. 
Rogers  defined  an  innovation  as  an  idea,  practice  or  object  that  is  perceived  as  new  by 
an  individual.  According  to  Rogers  (1986),  diffusion  is  the  process  by  which  an 
innovation  makes  its  way  through  a  society  or  specific  group  of  people.  The  process  of 
diffusion  includes  3  components:  1)  the  innovation,  2)  an  individual  or  other  unit  of 
adoption  who  knows  about  or  has  experience  with  the  innovation,  and  3)  a 
communication  channel  which  provides  a  means  of  information  exchange  between  the 
parties.  Communication  channels  include  two  types:  1)  mass  communication 
channels,  2)  interpersonal  channels. 

Diffusion  is  concerned  primarily  with  the  innovation  and  the  adoption  of  the 
innovation.  Following  the  introduction  of  the  innovation,  an  evaluation  process  takes 
place  on  the  part  of  the  consumer.  Diffusion  of  an  innovation  through  a  social  system 
is  determined  by  characteristics  of  the  adopter  and  the  perceived  value  of  the 
innovation. 

Early  adopters  have  been  found  to  possess  common  personal  characteristics.  In 
past  research  common  characteristics  of  early  adopters  have  been  found  in  the 
categories  of  socioeconomic  status,  communication  behavior,  and  personality  traits. 
More  specifically,  research  has  concluded  that  personal  characteristics  early  adopters 
have  in  common  are  their: 

1.  Education  Level 

2.  Social  Status 

3.  Social  Participation 
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4,  Cosmopolitan  Outlook 

5,  Mass  Media  Use 

6,  Personal  Communication 

7,  Degree  of  Innovation  Information  Seeking 

8,  Attitude  Toward  Change 

9,  Attitude  Toward  Risk 

10.  Aspirations 

11.  Attitude  Toward  FatcJism 

One  of  the  important  Individual  differences  m  length  of  the  Innovat  ion- 
decision  period  is  on  the  basis  of  adopter  category.  Early  adopters  have  a  shorter 
innovation-decision  period  than  later  adopter  The  first  individuals  to  adopt  a  new 
idea  (the  early  adopters)  do  so  not  only  because  they  become  aware  of  the  Innovation 
somewhat  sooner  than  their  peers,  but  also  because  they  require  fewer  months  and 
years  to  move  from  knowledge  to  decision  (Rogers  and  Shoemaker,  1971), 

Early  adopters  model  charge  agents.  A  c  hange  agent  is  a  professional  who 
influences  innovation-decisions  in  a  direction  deemed  desirable  by  a  change  agency 
(Rogers  and  Shoemaker,  1971).  Early  adopters  of  innovations  play  an  Important  role 
in  the  diffusion  process.  Their  initiative  to  adopt  an  innovation  to  observable  by  the 
later  adopters,  especially  the  early  majority,  This  observance  by  the  later  adoptera 
influences  their  adoption  and  thus  the  success  of  the  innovation. 

The  early  adopters  also  serve  as  a  filter  for  innovations.  If  the  early  adopters 
have  a  poor  perception  of  an  innovation,  the  innovation  will  be  Altered  out  of  the 
system  but  if  the  early  adopters  perceive  the  innovation  as  worthy  they  filter  the 
innovation  through  the  social  system,  Therefore,  early  adopters  initial  perceptions  of 
an  innovation  is  an  important  element  in  the  diffusion  process, 

According  to  Rogers  (1936),  adopters'  perceptions  of  an  innovation  are  also  a 
vital  element  in  the  diffusion  process.  lie  has  concluded  that  there  are  five  dominant 
characteristics  that  adopters  use  to  judge  the  value  of  an  innovation  Thesw 
characteristics  include; 

Relative  Advantage  -  the  degree  to  which  an  innovation  is  petcetved  au  being 
better  than  the  idea  it  supersedes,  The  relative  advantage  d  t\  new  Idea,  as 
perceived  by  members  of  a  social  system,  is  positively  related  to  Us  rate  of 
adoption, 

Compatibility  -  the  degree  to  which  an  innovation  is  perceived  being 
consistent  with  the  exteiing  values,  past  experiences  and  need;*  of  the 
receivers,  The  compatibility  of  a  new  idea,  as  perceived  by  membera  m  a 
social  system,  iA  positively    related  to  its  rate  of  adoption.. 
Triability  -  the  degree  to  which  an  innovation  may  hi  -mcperlwented  wi\h  on 
a  limited  basis,  Tti^  triability  of  a  new  idea,  as  perceived  by  members  of  a 
social  system,  is  positively    related  to  Ks  rate  of  adoption, 
Complexity  ■•  the  degree  to  which  an  innovation  is  penMfrcri  as  *tting 
relatively  difficult  to  understand  and  use,  The  complexity  erf  a  new  idea,  o» 
perceived  by  members  of  a  social  S3rstem.  is  negatively  related  to  itn  rate  of 
adoption, 

Observability  -  the  degree  to  which  the  re^u-ls  of  an  tarrwailfrn  are  vistole 

and  easily  communicated  to  others,  Vac  obsi  rvabfrity  of  a  new  idea,  as  perceived 

by  members  of  a  social  system,  is  positively  related  to  iis  rate  of  adoption. 

The  adoption,  implementation,  an  J  utilization  of  new  coTrimvvnicatiori 
technologies,  such  as  electronic  communication  networks  can  be  studied  effectively 
based  on  the  theoretical  framework  of  the  adoption/diffusion  Chewy.  Diffusion  oi  an 
innovation  is  determined  by  personal  characteristics  of  adopters  ix\  a  soaal  system  and 
their  perceived  value  of  the  innovation,  in  order  for  an  innovat  ion  io  be  successf  ully 
adopted  within  a  social  system  a  diffusion  process,  the  proctss  by  which  an  innovation 
makes  its  way  through  a  society  or  specific  group  of  people,  mu?3t  occur,  In  this  study, 
adoption  difTusion  theory  was  U3ed  as  the  thcorectical  base  to  study  early  adopters  use 
of  an  electronic  communication  network  for  teachers, 

The  electronic  communication  network  used  to  conduct  this  research  is  entitled 
the  Electronic  Educational  Exchange  (EEE),  The  college  of  Education  at  Iowa  State 
University  has  developed  an  electronic  network  to  bridge  the  gap  between  the  world  of 
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practice  in  the  classroom  and  education  faculty  at  the  university.  The  EEE  is  designed 
to  serve  the  following  purposes: 

L  Provide  a  convenient  method  for  the  exchange  of  ideas  betAveen  student 
teachers,  practicing  teachers  and  Iowa  State  faculty. 

A  Decrease  the  sense  of  isolation  often  encountered  by  student 
and  practicing  teachers. 

R  To  make  faculty  expertise  readily  available  to  student  and 
practicing  teachers. 

C.  To  increase  faculty  awareness  of  the  problems  frequently 
encountered  by  student  and  practicing  teachers. 

il.  Provide  telecommunications  experience  for  student  teachers,  practicing 
teachers,  and  Iowa  State  faculty. 


The  manner  in  which  educational  institutions  are  organized  generally  restricts 
teachers  to  their  classrooms,  which  limits  interaction  with  one  another.  Teacher 
isolation  is  currently  a  major  concern  within  the  field  of  education.  Many 
communication  formats  such  as  journals,  office  memos  and  school  calenders  serve  as 
methods  of  mass  communication,  but  alternative  methods  that  facilitate  interpersonal 
communication  must  be  examined. 

Electronic  communication  networks  are  relatively  new  systems  that  have  been 
proposed  as  one  method  to  decrease  teacher  isolation.  Although  research  has  begun  to 
indicate  that  electronic  networks  seem  to  be  a  "natural  fit"  to  meet  many  needs  in  the 
teaching  profession,  networks  haven't  been  adopted  and  diffused  on  a  large  scale  basis. 
There  is  a  need  to  collect  data  on  characteristics  of  early  adopters  to  better  understand 
the  people  and  why  they  use  electronic  networks  as  a  communication  device.  There  is 
also  a  need  to  understand  the  evolution  of  network  use.  In  order  to  determine  this 
information  a  theoretical  framework  must  be  implemented.  The  adoption/diffusion 
theory  provides  us  with  the  framework  needed  in  order  to  evaluate  the  adopters  of 
electronic  networks,  their  use  of  the  networks  and  the  evolution  of  electronic 
communication  networks.  The  data  will  assist  individuals  and  organizations  in  the 
implementation  and  management  of  electronic  communication  networks. 

Study  Procedures 

In  order  to  conduct  this  study,  a  list  of  the  most  frequent  EEE  users  in  the  Spring 
of  1990  was  generated.  The  most  frequent  or  heavy  users  of  the  system  were  defined  as 
early  adopters  of  the  innovation. 

A  total  of  35  subjects  were  selected.  The  subjects  consisted  of  4  Iowa  State 
professors,  4  Iowa  State  graduate  students,  4  Iowa  Stai^  student  teachers,  2  Iowa  State 
pre-servtce  teachers,  1  professor  outside  of  the  university,  and  20  educators  from  the 
state  of  Iowa  and  surrounding  states. 

A  questionnaire  was  used  in  this  study.  Research  has  suggested  that  personal 
characteristics  early  adopters  have  in  common  are:  education  level,  social  status, 
social  participation,  cosmopolitian  outlook,  mass  media  use,  personal 
communication,  degree  of  innovation  information  seeking,  aspirations  and  attitude 
toward  change,  risk  and  fatalism.  Therefore,  the  questionnaire  was  designed  to  collect 
data  in  these  areas.  In  addition,  research  suggests  that  perceptions  of  an  innovation 
directly  affect  the  diffusion  process.  An  additional  section  of  the  questionnaire 
allowed  the  early  adopters  to  rate  their  initial  and  current  perceptions  of  the  EEE  in 
five  categories.  The  categories  included  relative  advantage,  compatibility,  triability, 
complexity  and  observability. 


o 

ERIC 


3  200 
2»(i 


In  general,  the  results  of  the  study  with  respect  to  characteristics  yarly 
adopters  were  in  agreement  with  Rogers.  The  average  early  adopter  of  the  blectronfc 
Educational  Exchange  can  be  described  as: 

1.  possessing  a  Mast  ers  Degree  or  another  degree  oeyond  BA 

2.  having  an  income  of  $30,000  or  more 

3.  working  as  a  classroom  teachers 

4.  being  approximately  39  years-  old 

5  having  1 1  or  more  years  in  the  field  <tf  education 
6*.  contacting  teachers  outside  of  their  school  frequently 
7.  using  various  forms  of  media  frequently 
8  having  a  positive  attitude  wward  technology 
9.  having  an  attitude  that  risk  is  necessary  in  life 
10.  having  a  neutral  attitude  on  falattmn 

A  significant  finding  in  the  sbufy  involved  the  initial  arid  current  Perceptions  of 
the  Electronic  Ed>  .cational  Exchange.  The  subjects  were  asked  to  rate  the  EEE,  based on 
5  categories,  using  a  f've  point  scale  ranging  from  very  high  to  very  low.  The  mean  ol 
initial  VwcepUbn  was  2.41  indicating  that  the  users  felt  Uie  system  ranked  between 
average  a-d  high.  Later,  the  subjects  were  asked  to  rate  their  current  perception  of  the 
svstem  A  mean  of  1 .92  was  calculated  as  the  current  perception  of  the  system, 
indicating  that  the  users  had  a  significantly  higher  perception  cf  the  BEE  ^aii  wlien 
the  •  began  using  the  system.  Overall,  the  data  suggested  that  early  adopters  of  the  EEL 
were  receiving  the  services  they  were  seeking  from  the  system.  Results  are  repotted  m 
Table  ?.. 

Tahle  1.  Earlv  Adootei 's  InHlaiy  Curr@fr  .Eciceptlpns  of  E1CE  


PerceDHon  N  Mean         S.D.  T  J.-Tailed 

^ercepion  Probability 


Initial  Perception  2.41  .682 

29 

Current  Perception  "i.92 


.31  .001 


Message  content  and  wiUi  whom  individuals  commur ,  ?  itej  \;*s  ai  o  compiled 
and  evaluatec1  The  subjects  were  asked  to  comment  on  "with  vhea  they  .  wranunicate 
most' frequently"  and  "what  their  message  content  entails'  .  The  -ipoi  ses  were  stmilai 
Teachers,  Iowa  State  faculty,  and  graduate  students  were  the  most  common  iers 
mentioned  as  communication  partners  on  the  EEE.  Media  specialists  and  tne  system 
operator  were  a  so  mentioned. 

Message  content  was  also  similar  among  users;  content  categories  included 
research,  teaching  ideas,  educational  technology,  classroom  management  ano  personal 
messages.  This  suggested  mat  the  system  was  being  used  for  professional  development 
purposes  by  the  users. 


Education^  Implications 

Although  research  suggests  electronic  communication  networks  are  currently 
being  used  effectively  by  some  educaiors,  the  diffusion  process  of  such  networks  has  not 

been  ^{Jj;^  2^a^ffiests  that  early  adopters  play  a  vital  role  in  the  diffusion  process. 
They  serve  as  a  filter  for  the  network.  According  to  Rogers  (1956),  if  the  early  adopters 
have  a  poor  perception  of  an  innovation,  the  innovation  will  be  filtered  out  of  the 
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system  but  if  the  early  adopters  perceive  the  innovation  as  worthy  they  filter  the 
Innovation  through  the  social  system.  Early  adopters  of  the  EEE  were  asked  to  report 
their  initial  and  current  perception  of  the  system.  The  mean  of  the  initial  perception 
was  2.41  Indicating  that  the  users  felt  the  system  ranked  between  average  and  high. 
Later,  the  subjects  were  asked  to  rate  their  current  perception  of  the  system.  A  mean  of 
1.92  was  calculated  as  the  current  perception  of  the  system,  indicating  that  the  users 
had  a  significantly  higher  perception  of  the  EEE  than  when  they  began  using  the 
system.  Overall,  the  data  suggested  that  early  adopters  of  the  EEE  were  receiving  the 
services  they  were  seeking  from  the  system. 

Rogers  (1986)  stated  that  early  adopters  of  innovations  possess  certain 
characteristics  and  the  present  study  supports  his  findings.  In  general,  the  study  found 
early  adopters  of  the  Electronic  Educational  Exchange  to  possess  these  characteristics 
described  by  Rogers.  Therefore,  it  is  suggested  to  network  developers  that  they 
individually  seek  individuals  possessing  characteristics  similar  to  those  of  the  early 
adopters  of  the  EEE.  Their  use  will  increase  the  system's  observability  therefore, 
enhancing  the  diffusion  process. 
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2840  Sheridan  Road 
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Bitnet:  MHAN@NLU 

Researchers  using  both  qualitative  and  quantitative  data  are  concerned  with 
the  ways  in  which  information  can  best  be  organized,  analyzed  and  shared.  Software 
tools  for  analyzing  quantitative  data  have  been  widely  available  since  the  earliest 
days  of  computers,  but  software  that  assists  the  qualitative  researcher  in  the  analysis 
of  textual  data  is  relatively  new. 

During  this  session  we  will  demonstrate  a  HyperCard  program  fc  r  the 
Macintosh,  the  Data  Collector,  which  we  designed  for  our  own  research  and  have  now 
made  available  to  other  researchers  through  Intellimation.  The  Data  Collector  is  a 
tool  for  organizing  and  analyzing  textual  data  obtained  from  observations,  interviews, 
surveys,  and  other  documents.  The  program  also  includes  a  component  called  the 
BiblioStack  that  facilitates  note-taking  for  a  literature  review.  These  two  components 
may  be  used  together  or  independently. 

This  paper  describes  the  criteria  for  the  program's  development,  the  rationale 
for  using  HyperCard,  and  the  features  ot  both  the  Data  Collector  stack  and  the 
BiblioStack. 

Criteria  for  the  Program's  Development 

Wellman  and  Sim  (1990)  have  identified  several  criteria  for  evaluating 
software  meant  to  be  used  in  qualitative  research.  These  criteria  include  flexibility, 
preserving  the  richness  of  the  material,  ease  of  use,  and  analytic  power. 

Flexibility.  The  researcher  should  be  able  to  change  and  add  code  words 
because  objectives  and  understanding  change  during  the  analytic  process. 

Preserve  richness  of  material.  Wellman  and  Sim  recommend  that  textual 
analysis  software  allow  the  researcher  to  examine  the  material  in  context  and  to 
compare  similar  texts  for  analysis. 

Ease  of  use.  Researchers  should  be  able  to  use  the  software  without  having  to 
depend  on  an  expert  consultant. 

Analytic  power.  Textual  analysis  software  should  help  to  identify  classes  of 
concepts,  do  complex  Boolean  searches,  and  provide  output  in  a  form  that  facilitates 
analysis. 

In  designing  this  program  we  have  made  every  effort  to  take  these  criteria  into 
consideration. 

Why  HyperCard? 

Both  the  Data  Collector  and  the  BiblioStack  are  written  in  HyperCard,  an 
authoring  program  for  the  Macintosh  that  allows  access  to  a  large  body  of  information 
in  a  variety  of  different  ways.  An  important  characteristic  of  HyperCard  is  that  any 
piece  of  information  can  be  linked  to  any  other  piece  of  information,  creating  a  web  of 
relrted  ideas.  Because  HyperCard  facilitates  the  linking  jf  many  disparate  pieces  of 
text,  it  seemed  to  us  an  ideal  environment  for  organizing  and  analyzing  qualitative 
data. 
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Features  of  the  Data  Collector  Stack 
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The  Data  Collector  stack  has  as  its  backbone  a  single  card  where  you  can  enter 
any  kind  of  textual  data  (Figure  1).  Data  may  be  typed  into  a  scrolling  Notes  field  or 
imported  into  the  Notes  field  from  a  word  processor,  m  addition  to  the  Notes  field, 
there  are  fields  for  the  type  of  data  (interview,  observation,  survey,  historical 
document),  general  site  description,  the  specific  setting,  the  name  of  the  researcher  and 
the  date. 

Pftplpan  searching.  HyperCard^  normal  Find  function  has  been  enhanced  to 
include  Boolean  searching  using  multiple  and's  and  or's.  The  Find  and  Find  Again 
options  in  the  Utilities  menu  locate  all  occurrences  of  a  specified  word  or  phrase.  The 
results  of  the  search  may  be  viewed  on  the  screen  or  copied  to  a  specific  topic  card 
created  by  the  researcher. 

Coding.  The  Coding  menu  allows  you  to  code  a  document,  as  well  as  to  add, 
change  or  delete  code  words  during  the  analysis  process.  First,  enter  your  own  code 
words.  Then  you  click  on  a  code  word  in  your  list  and  click  on  the  paragraph  to  be  coded. 
The  code  word  is  placed  within  brackets  at  the  beginning  of  that  paragraph. 

Copying  text  to  fl  toPir  card  To  facilitate  the  data  reduction  process  and  in 
order  to  construct  themes  ?  i  domains  for  further  analysis,  text  may  be  copied  to  topic 
cards,  thereby  reducing  th  data  to  smaller,  more  manageable  divisions  for  further 
analysis.  To  do  this,  highlight  the  selected  text  using  the  mouse  and  click  the  Copy 
Text  button.  You  choose  whether  to  copy  text  to  an  existing  topic  card  or  to  a  newly 
created  topic  card.  When  the  text  is  copied  to  the  card,  information  identifying  the 
source  of  the  data  accompanies  it.  To  continue  the  data  reduction  process,  the  text  on  the 
topic  cards  can  be  coded  and  even  copied  to  a  new  topic  card. 

Comparative  viewing.  The  Compare  Cards  button  allows  you  to  view  two  cards 
on  the  screen  simultaneously.  To  mark  cards  for  comparative  viewing,  click  on  {he 
dotted  corner  on  the  card;  the  corner  will  turn  down.  The  marked  cards  are  then  placed 
in  a  separate  stack  in  another  window  on  the  screen  for  easy  comparison  with  the 
original  stack.  The  ability  to  compare  documents  aids  in  the  triangulation  process. 
While  in  this  mode,  text  from  either  window  may  be  copied  to  new  or  existing  topic 
cards. 

Importing  and  exporting  text.  The  Export  and  Import  items  in  the  Utilities 
menu  provide  a  means  for  transferring  text  files  to  and  from  word  processing  programs. 

Features  of  the  BiblioStack 

The  second  component  of  the  Data  Collector  program  is  the  BiblioStack,  which 
facilitates  note-taking  for  a  literature  review.  From  the  BiblioStack  you  can  create 
reference  cards  containing  information  about  a  citation  along  with  any  number  of  note 
cards  related  to  that  citation. 

Reference  cards.  The  reference  card  for  each  bibliographic  entry  includes 
standard  bibliographic  information,  a  field  for  key  words,  and  a  field  for  the  citation 
-  jstract  (Figure  2).  The  complete  citation  is  placed  in  one  field,  thus  you  can  enter  it 
using  either  APA,  Chicago,  MLA  or  any  other  style.  Citations  can  be  directly  exported 
to  a  word  processing  program  to  create  a  bibliography  or  list  of  references.  No  retyping 
is  needed  if  the  citation  has  been  entered  into  the  BiblioStack  using  the  proper  style, 
although  additional  formating  may  be  necessary. 

Note  cards.  Each  reference  card  has  associated  with  it  any  number  of  note  cards. 
Each  note  card  contains  a  field  for  notes  and  a  field  for  key  words  relevant  to  the 
particular  note  (Figure  3). 

Boolean  searching.  You  can  perform  Boolean  searches  to  find  any  word  or 
phrase  in  a  reference  card  or  note  card.  The  results  of  the  search  are  viewed  on  the 
screen  and  may  be  printed  out. 
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IjjpjiSLSlflfikS-  Reference  cards  meeting  a  particular  search  criteria  can  be  marked 
and  copied,  with  all  their  note  cards,  to  a  new  independent  topic  stack.  In  this  way  you 
can  create  separate  bibliogtaphic  stacks  to'  different  research  projects  from  your 
original  stack. 

Conclusion 

Although  originally  designed  for  our  own  use,  we  believe  the  Data  Collector 
demonstrates  the  viability  of  using  technology  to  meet  the  needs  of  the  qualitative 
researcher.  It  is  important  to  remember,  though,  that  the  software  is  simply  a  tool  for 
organising  and  managing  the  data;  the  researcher  must  still  do  the  analysis.  The 
researcher  must  develop  the  relevant  coding  categories,  identify  emerging  themes,  and 
support  his  or  her  interpretations  from  the  data.  However,  software  tools  such  as  this 
one  can  simply  the  mechanical  aspects  of  that  task  considerably. 
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The  Effects  of  Integrating  a  Learning  Strategy  in  CBI 

Tutorial  courseware  allegedly  enacts  a  dialog  between  the  program  designer 
and  the  learner.  According  to  Soulier  (1988),  "dialog  frames"  present  information 
to  the  learner,  as  well  as  carry  out  an  interactive  dialog/feedback  between  the 
learner  and  the  computer.  Soulier  borrows  the  terminology  of  "dialog  frames" 
from  Alfred  Bork  (1980),  who  defined  dialog  as  "a  'conversation1  between  a  student 
and  a  teacher,  wheve  the  student  is  at  the  computer  display  and  the  teacher  is 
conducting  the  dialog  through  the  medium  of  a  computer  program"  (p.  15). 

Soulier  (1988)  explains  that  dialog  frames  in  computer-based  instruction  (CBI) 
usually  have  three  parts:  rules,  examples  and  interactions.  The  information  or 
subject  matter  is  usually  presented  in  the  first  two  parts.  The  focus  of  this  study  is 
concerned  with  the  third  part,  the  practice  component,  or  as  Soulier  refers  to  it,  the 
interaction  function, 

According  to  Alessi  and  Trollip  (1985),  the  most  common  method  of  CBI 
interaction  is  for  the  computer  to  pose  questions  the  learner  must  answer. 
Questions  serve  a  number  of  purposes.  They  keep  the  student  attentive  to  the  lesson, 
force  the  student  to  practice,  and  assess  the  student  s  recall  and  comprehension.  In 
addition,  student  responses  to  the  questions  determine  the  sequencing  for  the 
remainder  of  the  lesson. 

Questions  can  be  categorized  into  two  basic  types:  selected-response  and 
constmcted-response.  Selected-response  questions  are  those  for  which  the  student 
chooses  the  correct  response  or  responses  from  a  list  of  two  or  more  options.  These 
include  true/false,  matching,  multiple-choice,  and  marking.  Multiple-choice 
items  are  the  most  common  type  of  selected-response  questions  used  in  CBI  (Alessi 
&  Trollip,  1985).  Constmcted-response  questions  are  open-ended  questions  that 
require  the  student  to  produce  rather  than  select  a  response.  The  three  mqjor  types 
of  constmcted-response  questions  are  completion,  short-answer,  and  essay.  In 
CBI,  the  most  common  are  completion  and  short-answer  questions. 

In  a  chapter  of  a  recent  book  on  designs  for  microcomputer  courseware, 
Jonassen  (1988)  states  that  one  of  the  problems  with  most  tutorial  courseware  is  that 
the  level  of  mental  processing  required  of  learners  is  too  shallow.  He  claims  this 
weakness  is  due  to  the  multiple-choice  and  short-answer  questions  predominantly 
used  in  CBI  interactions.  He  states  that  the  nature  of  the  interactivity  is  usually 
recognition  and  recall  of  information  presented  on  the  previous  screen,  with  little, 
if  any,  attempt  to  relate  the  information  to  prior  knowledge,  so  that  it  will  be 
meaningful  and  therefore  retrievable  by  the  learner.  Jonassen  (1988)  states, 
Tutorial  courseware  is  basically  a  mis-application  of  the  programmed 
learning  model  of  instructional  design,  which  has  been  the  dominant 
paradigm  in  the  field  of  educational  technology  for  nearly  three  decades.  Since 
the  programmed  learning  model  is  easily  contused  with  the  procedure  or 
technique  of  programmed  instruction,  it  is  better  conceptually  defined  by  the 
mathemagenic  hypothesis. 

Mathemagenic  behaviors  are  "those  student  activities  that  are  relevant  to 
the  achievement  of  specified  instructional  objectives  in  specified  situations  or 
places,"  that  is,  those  which  "give  birth  to  learning"  (Rothkopf,  cited  in 
Jonassen,  1988).  These  behaviors,  according  to  the  hypothesis,  can  be  controlled 
or  manipulated  by  specific  design  attributes  of  instruction.  The  form  or 
structure  of  instruction  or  the  activities  stimulated  by  it  induce  the  necessary 
cognitive  operations  to  produce  the  desired  learning.  .  . .  The  purpose  of 
mathemagenic  activities,  such  as  inserted  questions  (the  basis  of  programmed 
learning),  is  to  control  the  way  in  which  information  is  transformed  and 
encoded  into  memory.  It  is  therefore  a  reductive  approach  to  learning,  which 
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regards  learr  <v.  #  ,  active  performers  whose  mental  behavior  should  be  strictly 
controlled  b*    ><t '  fivities  imposed  by  the  lesson,  (p.  152) 
Jonassen  contiw<>  *  by  stating  tbr*,  one  of  the  problems  with  computer 
applications  of  the  mathemagenic  model  is  that  the  level  of  processing  normally 
produced  by  mathemagenic  behaviors  (especially  programmed  learning)  is  too  low 
or  shallow. 

To  solve  the  problem,  Jonassen  (1988)  recommends  tutorial  courseware  be 
developed  based  on  a  more  constructive  conception  of  learning,  that  is,  instruction 
which  focuses  more  on  cognition,  requiring  deeper  levels  of  mental  processing. 

In  order  to  increase  the  level  of  mental  processing,  Jonassen  (1988) 
recommends  that  tutorial  courseware  should  enhance  meaning  of  course 
information  by  stimulating  the  learners  to  call  up  and  apply  what  they  already 
know.  He  suggests  that  this  deeper  processing  can  be  achieved  by  incorporating 
learning  strategies  into  the  courseware.  Thesu  learning  strategies  sometimes 
referred  to  as  cognitive  strategies,  are  mental  operations  or  procedures  that 
represent  a  wide  range  of  cognitive  activities  including  underlining  mam  ideas, 
generating  examples  or  analogies,  summarising,  outlining,  etc.,  which  the 
student  may  use  to  acquire,  retain,  and  retrieve  different  kinds  of  knowledge 
(Rigne; .,  1978\  They  are  designed  to  increase  the  aumber  of  "links"  between 
presented  information  and  existing  knowledge  in  order  to  enhance  retention  and 
to  allow  the  student  to  process  information  generatively  (Wittrock,  1974).  They  are 
always  performed  by  the  learner-at  the  initiation  of  the  student  or  the  instructional 
system. 

According  to  Wei n stein  and  Mayer  (1986),  in  recent  years  increased  attention 
has  been  placed  on  the  role  of  the  learner  as  an  active  participant  in  the  teaching- 
learning  act.  Many  articles  suggest  that  the  effects  of  instruction  depend  partly  on 
prior  knowledge,  and  the  learner's  active  cognitive  processing  during  the 
learning  process  (Anderson,  Spiro  &  Montague,  1977;  Cook  &  Mayer,  1983; 
Dansereau,  1985;  Jones,  Amiran  &  Katims,  1985;  Mayer,  1984;  Ryan,  1981; 
Weinstein,  1978;  Weinstein  &  Underwood,  1985;  Wittrock,  1974,  1978).  Wittrock's 
(1974)  generative  learning  hypothesis  assumes  that  when  faced  with  instructional 
stimuli,  learners  construct  and  assign  meaning  to  the  stimuli  on  the  basis  of  prior 
learning.  The  meaning  generated  by  learners  for  information  they  receive  is 
individual  and  cannot  be  controlled  by  the  author.  According  to  Wittrock  (cited  in 
Jonassen,  1988),  learning  is  not  "a  passive  reception  of  someone  else's 
organizations  and  abstractions"  (p.  153).  Instead,  learning  is  an  active, 
constructive  process.  Jonassen  (1985)  states, 

The  purpose  of  generative  strategies  is  to  provide  learners  with  active, 
constructive  skills  for  proactively  transferring  prior  knowledge.  Meaning 
(knowledge,  if  you  prefer)  is  learner-constructed,  not  media-controlled.  These 
[learning]  strategies,  such  as  paraphrasing,  generating  questions,  and 
imaging,  are  all  individual  processes  fc  *  constructing  meaning, 
(p.  31) 

Jonassen  (1985)  explains  that  microcomputers  are  especially  amenable  to  the 
inclusion  of  learnir  J  strategies  because  they  can  accept,  store,  and  v    update  a 
variety  of  input,  'jid  they  can  insist  on  a  response  before  allowing  the  learner  to 
proceed;  something  which  traditional  print  media  cannot  do.  For  instance,  after 
presenting  the  information,  the?  computer  system  could  direct  the  learner  to  go 
through  a  particular  learning  strategy,  wi)h  the  omputer  functioning  as  an 
"electronic  notebook."  The  learner  c&n  key  in  a  response  which  the  computer 
evaluates  foi  quantity  or  existence  of  key  concepts  and  sometimes  manipulates 
through  rearranging  or  mapping  and  then  stores  for  use  by  learners  as  a  review  or 
retrieval  strategy.  Jonassen  (1988)  believes  that  assessing  user  input  will  ensure 
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higher  levels  of  processing  than  the  multi-option  recognition  or  recall  tasks 
included  in  most  tutorial  courseware. 

Jonassen  (1988)  recommends  that  "the  simp]  tit  method  for  integrating 
[learning]  strategies  in  courseware  is  to  replace  the  adjunct,  mathemagenic 
activities  that  are  normally  included  as  practice  in  courseware  with  specific 
information  processing  or  perhaps  metaleaming  strategies19  (p.  160).  More 
specifically  he  suggests: 

Fo:  example,  rather  than  inserting  multiple-choice  questions  to  test  immediate 
recall  or  comprehension  of  information  in  a  program^  you  might  periodically 
insert  any  of  the  following  directions:  Summarize  in  your  own  words  the  ideas 
presented;  recall  and  record  key  i  leas  and  use  them  to  create  analogies, 
outlines,  or  cognitive  maps;  draw  a  picture  or  generate  a  mental  image  of  the 
subject  matter;  or  list  the  implications  of  the  material  that  you  are  studying, 
(p.  160) 

In  discussing  the  rationale  for  replacing  multiple-choice  and  short-answer 
questions  in  CBI  interaction  frames,  Jonassen  (1988)  explains  that  "only  deeper, 
semantic  processing  of  information  requires  the  learner  to  access  prior  knowledge 
in  order  to  interpret  new  material*  (p.  153).  He  makes  the  point  that  integrating 
cognitive  strategies  into  CBI  interactions  will  facilitate  higher  level  learning 
outcomes,  such  as  comprehension  and  understanding,  and  will  also  increase  the 
amount  of  information  that  is  recalled.  Because  learners  assign  more  meaning  to 
the  information  presented  during  the  tutorial,  they  will  be  able  to  remember  more. 
Jonaesen  continues, 

It  10  exactly  this  level  of  meaningful  learning  that  is  most  frequently  missing 
from  tutorial  types  of  courseware.  The  emphasis  is  on  practice  of  associations 
in  working  memory  based  c.ily  on  information  recently  presented  in  the 
courseware.  In  interacting  with  the  tutorial  courseware,  learners  are  too 
seldom  required  to  access  prior  knowledge  in  order  to  interpret  the  information 
that  is  presented  (p.  153) 

The  primary  purpose  of  this  study  was  to  examine  the  effects  of  replacing 
multiple-choice  questions  in  tutorial  courseware  practice  interactions  with  a 
learning  strategy  (learner-generated  summaries)  on  immediate  recall.  The 
study  was  designed  to  investigate  empirically  the  approach  recommended  by 
Jonassen  (1988)  to  integrate  learning  strategies  by  replacing  the  traditional 
multiple-choice  questions  in  CBI  interactions. 

Although  several  studies  within  the  last  few  years  have  investigated  the  effects 
of  integrating  individual  learning  strategies  in  instructional  systems  and 
developing  instructional  systems  that  train  learners  how  to  use  a  particular 
learning  strategy  (Weinstein  &  Mayer,  1985;  Dansereau  et  al.,  1979;  (Weil,  1978; 
0*Neil  &  Spielberger,  1979),  the  practice  of  integrating  learning  strategies  in  CBI 
interactions  has  not  received  much  attention  in  the  research  literature.  The  few 
studies  which  have  been  conducted  had  a  different  focus  than  the  present  study.  For 
example,  these  studies  either  dealt  with  a  different  learning  outcome  tJhan 
immediate  recall  or  the  study  tried  to  validate  a  different  type  of  learning  strategy 
(for  example,  see  Eucker,  1984;  Allen,  1982;  Wilshire,  1990).  The  findings  from 
these  studies  did  not  support  the  hypothesis  that  integrating  a  learning  strategy  in 
CBI  would  be  more  effective  in  facilitating  learning  than  the  traditional  approach 
used  by  the  comparison  group(s). 

Several  studies  have  examined  different  aspects  of  student-generated 
summaries  (Annis,  1985;  Brown,  Campione,  &  Day,  1981;  Day,  cited  in  Brown, 
Campione,  &  Day,  1981;  Ballestevos,  1986);  however,  none  have  examined  the 
specific  effects  of  integrating  this  strategy  in  CBI  interactions  on  immediate 
recall. 

The  following  hypotheses  and  research  questions  were  addressed  in  this  study: 
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Hypotheses 

1.  Learner-generated  summaries  in  tutorial  courseware  will  be  More  effective  in 
facilitating  learners1  ability  to  inn.  diately  recall  information  than  multiple- 
choice  interactions. 

2.  A  system-generated  summary  provided  to  learners  (as  a  form  of  feedback) 
after  they  generate  their  own  summaries  will  be  more  effective  in  facilitating 
learners*  ability  to  immediately  recall  information  than  not  providing  a 
syst  *n-generated  summary. 

Research  Questions 

1.  Will  students  who  are  required  to  generate  summaries  take  significantly  more 
time  to  complete  the  intervention  than  students  who  respond  to  multiple-choice 
interactions? 

2.  Will  students  who  are  provided  a  system-generated  summary  type  fewer  words 
in  their  summaries  than  students  who  are  not  provided  a  system-generated 
summary? 

3.  Will  typing  rather  than  writing  the  learner-generated  summaries  affect  the 
quantity  or  quality  of  the  summaries? 

4.  To  what  degree  will  students  use  self-initiated  learning  strategies  during  the 
intervention? 

Method 

Subjects 

Subjects  were  obtained  from  an  undergraduate  computer  course  (Computers  in 
Education)  offered  in  the  College  of  Education  at  Brigham  Young  University  in 
Provo,  Utah.  They  were  required  to  complete  the  intervention  for  this  study  as  one 
of  their  assignments  in  the  course.  Thirty-five  students  participated  in  the  study: 
thirty-two  females  and  three  males. 

Materials 

The  intervention  for  all  treatment  groups  consisted  of  an  interactive  videodisc 
tutorial  on  the  subject  matter  of  developmental  biology,  developed  by  WICAT 
Incorporated  in  1979,  under  a  grant  from  the  National  Science  Foundation 
(Bunderson,  Olsen  &  Baillio,  1981).  The  hardv  are  consisted  of  a  Macintosh-based 
interactive  videodisc  system. 

The  instruction,  which  was  developed  fo:  university-level  students,  dealt 
primarily  with  verbal  information  learning  outcomes.  This  explanative  material 
(Mayer,  1989)  consisted  of  an  introduction  to  basic  developmental  biology  concepts 
such  as  DNA,  RNA,  genes,  chromosomes,  etc.  It  was  selected  for  several  reasons. 
First,  the  researcher  believed  that  most  students  had  very  little  prior  knowledge 
about  the  content,  which  was  desirable  in  determining  whether  a  learning  effect 
occurred  as  a  result  of  the  treatment.  Second,  the  instruction  on  the  videodisc  had 
been  developed  using  the  traditional  type  of  CBI  interactions:  an  instructional 
segment  followed  by  multiple-choice  questions.  Finally,  since  the  content  area 
was  not  related  to  the  course  in  which  r  S-donts  were  enrolled,  the  researcher 
believed  the  students  would  not  make   itensive  use  of  self-initiated  learning 
strategies  as  they  completed  the  instruction. 

The  interaction  frames  and  the  courseware  that  controlled  the  videodisc  were 
developed  on  the  computer  by  use  of  the  authoring  program  The  Best  of  Course  of 
Action^  developed  by  Authorware,  Incorporated  (Authorware,  1987). 
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Research  Design 

The  research  was  conducted  using  a  Pretest-Posttest  Comparison  Group 
Design.  The  students  were  randomly  assigned  to  one  of  three  treatment  groups. 
The  instruction  delivered  during  the  intervention  was  the  same  for  all  three 
treatment  groups-four  modules  consisting  of  videodisc  motion  sequences  and 
several  still  frames  concerning  developmental  biology.  Students  in  Group  1 
answered  multiple-choice  questions  after  each  instructional  module.  In  Group  2, 
students  were  asked  to  generate  a  written  summary,  which  they  typed  in  their  own 
words  on  the  computer,  about  the  information  presented  in  each  module.  Students 
in  Group  3  were  also  asked  to  type  a  written  summary  of  the  information  just 
presented  to  them;  however,  after  they  wrote  their  summary,  they  were  provided 
feedback  in  the  form  of  a  system-generated  summary  which  summarized  all  of  the 
main  points  in  the  module  and  were  told  to  compare  the  system-generated 
summary  with  their  own  to  determine  if  they  had  left  out  any  important  ideas. 

The  independent  variable  was  the  type  of  learner  response  required  during  the 
CBI  interaction  following  the  instructional  modules.  The  dependent  variable  for 
the  hypotheses  was  the  posttest  score. 

Instruments 

A  pretest  was  developed  to  assess  the  preexisting  knowledge  of  students  on  the 
content  of  developmental  biology  in  order  to  determine  if  the  three  treatment  groups 
were  equivalent  as  a  result  of  random  assignment  to  groups.  A  posttest  was 
developed  using  items  selected  from  a  posttest  developed  by  WICAT  Inc.!  based  on 
the  information  presented  in  the  videodisc.  The  test  consisted  of  multiple-choice, 
matching,  true/false,  and  short-answer  test  items  which  assessed  recall  learning 
outcomes. 

Internal  consistency  reliability  measures  were  computed  on  the  test  scores  to 
assess  the  reliability  of  the  results.  The  reliability  (Cronbach  Alpha)  coefficients 
for  the  pretest  and  posttest  scores  were  .76  and  .72,  respectively. 

Procedure 

The  researcher  administered  the  intervention  for  each  student  individually 
during  a  one-hour  time  slot.  When  the  student  arrived,  the  researcher  briefly 
explained  the  nature  of  the  study  and  then  i  ^quested  him  or  her  to  sign  a 
participation  consent  form  and  complete  the  pretest.  The  researcher  then  gave  the 
student  a  brief  overview  on  how  to  use  the  instructional  system. 

After  the  orientation,  the  students  wen,  presented  a  module  of  instruction 
consisting  of  several  still  frames  and  a  motion  sequence.  The  students  were  then 
asked  to  either  answer  multiple-choice  questions  or  generate  a  summary  of  the 
information  just  presented.  Students  in  Group  1  who  selected  the  correct  answer 
were  given  a  short  statement  affirming  the  correct  selection.  Students  who  selected 
the  wrong  answer  were  told  their  answer  was  incorrect  and  then  given  the  correct 
answer.  Students  in  Group  2  were  asked  to  generate  a  summary  of  the  material  just 
presented.  After  the  student  typed  his  or  her  summary,  the  instructional  system 
presented  the  next  module.  Students  in  Group  3  were  given  the  same  instructions  as 
those  in  Group  2;  however,  immediately  after  the  students  typed  their  summary,  the 
system  presented  a  complete  summary  of  all  the  main  ideas  presented  as  a  form  of 
feedback.  Students  were  asked  to  compare  their  summaries  to  tho  system- 
generated  summary  to  determine  if  they  had  left  out  any  important  information  or 
concepts.  This  pattern  continued  f>r  the  remaining  three  module**.  Students  in 
Group  1  answered  a  total  of  20  multiple-choice  practice  items,  and  students  in 
Groups  2  and  3  generated  four  summaries  during  the  entire  intervention. 

After  the  students  completed  the  intervention,  they  took  the  posttest.  Students 
took  an  average  of  60  minutes  to  complete  the  entire  process. 
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Datu  Analysis 

The  hypotheses  were  analyzed  using  a  planned,  orthogonal  comparison 
statistical  design.  Information  related  to  the  first  two  research  questions  (minutes 
of  elapsed  time  and  average  number  of  words  written  in  the  learner-generated 
summaries  groups)  was  analyzed  for  the  three  treatment  groups  using  one-way 
analysis  of  variance.  The  last  two  research  questions  were  analysed  using 
descriptive  statistics. 

Results 

Of  the  35  subjects,  12  were  in  Group  1,  which  answered  multiple-choice 
questions,  11  were  in  Group  2  which  generated  jummaries  without  feedback,  and  12 
were  in  Group  3  which  generated  summaries  and  received  feedback.  Means, 
standard  deviations,  and  ranges  were  computed  on  (a)  the  number  of  correct 
answers  on  the  pretest  (20  points  possible),  (b)  number  of  correct  answers  on  the 
posttest  (33  points  possible),  (c)  instructional  time,  and  (d)  number  of  words  in 
student-generated  summaries  (for  Groups  2  and  3  only).  Table  1  summarizes  these 
computations. 

Table  1 

Group  Means,  Standard  Deviations,  and  Ranges  for  Pretest  Scores,  Posttest  Scores, 
Instructional  Time,  and  Number  of  Words  in  Student-Generated  Summaries 


Number  of 

Treatment      Pretest  Posttest        Instructional       Words  in 

Group  Scores  Scores  Time  (minutes)  Summaries 


Group  1  (Multiple-choice  items) 

Mean        8.3               24.3                25.0  NA 

SD            3.1                2.5                  5.1  NA 

Range      2-12              1828               19-33  NA 

Group  2  (Student-generated  summaries  without  feedback) 

Mean        8.2               23.2                35.6  205.0 

SD            4.6                 3.7                 12.0  74.2 

Range      1-14              17-28              2064  87-344 

Group  3  (Student-generated  summaries  with  feedback) 

Mean        8.9               24.0                34.9  236.2 

SD            4.2                 4.1                  8.5  116.6 

Range      2-15              16*30               17-53  77-504 
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One-way  analysis  of  variance  revealed  that  the  differences  between  means  on 
the  pretest  scores  were  not  statistically  significant,  indicating  that  the  three  groups 
were  equivalent  in  preexisting  knowledge  about  the  material  presented  in  the 
instruction,  P  (2, 32)  =  ,11,  p  =  .896. 

The  first  hypothesis  (learner-generated  summaries  groups  will  score  higher 
than  the  multiple-choice  group)  was  tested  using  a  complex,  orthogonal,  planned 
comparison  analysis  (Keppel  &  Zedeck,  1989),  in  which  the  average  of  the  means 
for  the  two  learner-generated  summaries  groups  (Groups  2  &  3)  was  compared  with 
the  mean  of  the  multiple-choice  practice  group  (Group  1).  Results  from  the  analysis 
indicated  no  significant  treatment  effects,  T  (32)  =  .53,  p  =  .60. 

The  second  hypothesis  (Group  3  will  score  higher  than  Group  2)  was  tested  using 
a  pairwise,  orthogonal,  planned  comparison  analysis,  in  which  the  mean  of  the 
learner-generated  summaries  without  feedback  group  (Group  2)  was  compared 
with  the  mean  of  the  learner-generated  summaries  with  feedback  group  (Group  3). 
Results  from  the  analysis  indicated  no  significant  differences  between  the  two 
groups,  T  (32)  =  .57,  p  s  .58. 

With  regard  to  the  question  of  instructional  time,  Table  1  shows  that  students  in 
Group  1  spent  an  average  of  10  minutes  less  than  students  in  Groups  2  and  3  during 
the  instruction.  One-way  analysis  of  variance  resulted  in  a  significant  difference 
between  the  means  of  the  treatment  groups,  F  (2,32)  s  5.3,  p  =  .01.  Results  of  a 
Tukey-HSD  post  hoc  comparison  analysis  (p  <  .05)  indicated  that  Groups  2  and  3 
differed  significantly  from  Group  1. 

The  second  research  question  sought  to  determine  if  there  was  a  significant 
difference  in  the  number  of  words  generated  by  the  students  in  Group  2  and  Group  3, 
based  on  the  concern  that  students  in  Group  3  might  minimize  the  words  they  wrote 
in  their  self-generated  summaries  as  a  result  of  knowing  they  would  be  presented  a 
system-generated  summary  after  typing  in  their  summary.  Table  1  shows  that 
Group  3  generated  an  average  of  30  more  words  than  students  in  Group  2.  A  one- 
way analysis  of  variance  showed  that  the  difference  between  means  was  not 
statistically  significant,  F  (1,  21)  =  .572,  p  =  .458. 

The  third  research  question  examined  whether  typing  rather  than  writing 
affected  the  quality  of  the  students1  summaries.  To  answer  this  question,  the 
researcher  asked  students  who  generated  their  own  summaries  if  they  felt  typing 
versus  writing  their  summaries  affected  the  quality  and  quantity  of  those 
summaries.  Of  the  23  students  who  responded,  three  students  indicated  that  they 
would  have  written  more  and  their  summaries  might  have  been  of  a  higher  quality 
had  they  been  allowed  to  write  the  summaries  on  a  piece  of  paper.  All  other  students 
indicated  that  typing  was  not  a  factor  in  generating  their  summaries*  Twenty 
students  said  they  would  prefer  to  type  their  summaries  if  given  a  choice,  whereas 
three  students  indicated  they  would  preier  to  write  their  summaries. 

The  fourth  research  question  was  critical  to  the  fnal  study:  If  students  used 
extensive  3elf-initiated  learning  strategies  in  addition  to  those  designed  in  the 
interventions,  the  results  would  perhaps  be  influenced  more  by  learner  variance 
than  by  treatment  variance.  The  students  were  interviewed  and  asked  what  self- 
iniuated  learning  strategies,  if  any,  they  used  as  they  progressed  through  the 
instruction.  The  results  as  summarized  in  Table  2,  show  that  66  percent  of  the 
students  used  rehearsal  strategies  which  consist  of  repeating  more  than  once  in 
their  minds  the  material  presented  to  them.  However,  as  Table  2  shows,  few 
students  used  any  learning  strategies  other  than  the  rehearsal  technique 
strategies. 
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Table  2 

Self-initiated  Learning  Strategies  Used  by  Students 


Response 
Categories 


Number  of 
Respondents 


Percentage  of 
Respondents 


Rehearsal 


Organizational  Strategy 


Mnemonics 


Mental  Imagery 


Note  Taking 


23 
8 
7 


2 


66% 
23% 
20% 
9% 
6% 


Note:  These  percentages  do  not  add  up  to  100  percent  because  students  could 
give  more  than  one  response. 


Discussion 

The  results  of  this  study  did  not  support  the  first  hypothesis:  replacing  multiple- 
choice  questions  with  learner-generated  summaries  in  tutorial  courseware  did  not 
result  in  learners  recalling  significantly  more  of  the  information  presented 
during  the  instruction.  In  addition,  the  results  show  that  students  in  the  learner- 
generated  summary  groups  took  significantly  more  time  to  complete  the 
intervention  than  students  in  the  multiple-choice  questions  group. 

It  was  also  hypothesized  that  providing  a  system-generated  summary  to 
learners  after  they  generated  their  own  summary  would  be  more  effective  in 
facilitating  learners1  ability  to  immediately  recall  information  than  not 
providing  a  system-generated  summary.  The  results  did  not  support  this 
hypothesis. 

What  are  some  of  the  theoretical  implications  of  these  findings?  First,  because 
of  the  small  number  of  subjects  in  each  group,  learner  variance  could  have  had  a 
significant  influence  on  the  results.  The  researcher  plans  to  continue  this  line  of 
research  with  a  larger  sample  size.  Second,  for  the  particular  recall  learning 
outcome  assessed  in  this  study,  students  may  need  the  practice  items  of  multiple- 
choice  questions  to  provide  the  necessary  repetition  required  for  immediate  recall 
(Gagne,  1985;  Gagne,  Briggs  &  Wager,  1988).  Third,  the  multiple-choice  response 
items  may  have  highlighted  the  key  knowledge  elements  presented  in  the 
instruction  which  focused  the  learners1  attention  on  the  information  that  was 
really  important.  Fourth,  another  theoretical  issue  which  may  have  had  some 
influence  on  the  results  of  this  study  is  the  degree  of  instructional  alignment 
between  the  learning  strategy  and  the  learning  outcomes  (Cohen,  1987).  Perhaps 
using  learner-generated  summaries  was  not  the  most  appropriate  strategy  to  use 
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for  the  particular  learning  outcome  measured.  Learner-generated  summaries 
may  have  facilitated  other  higher-level  learning  outcomes  that  were  not  measured 
in  this  study,  such  as  understanding,  transfer,  synthesis,  etc.  Finally,  another 
possible  explanation  which  may  have  influenced  the  results  of  this  study  was  the 
degree  to  which  students  wrote  effective  summaries.  Perhaps  the  subjects  in  this 
study  were  much  more  familiar  with  responding  to  short-answer  and  multiple- 
choice  items  and  hadnvt  received  much  practice  in  writing  summaries. 

The  results  of  this  study  show  that  replacing  multiple-choice  questions  in 
tutorial  coursewp"*  with  learner-generated  summaries  does  not  facilitate  the 
specific  learning  utiime  of  immediate  recall  for  the  specific  material  used  in  this 
study.  In  fact,  replac*.;g  multiple-choice  questions  with  learner-generated 
summaries  in  CBI  interactions  reduced  the  efficiency  of  the  learning  process, 
since  students  who  generated  their  own  summaries  took  significantly  longer  to 
complete  the  instruction. 
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Introduction 

The  other  two  contributors  are  focusing  on  Uie  physical  facilities  of  the  environment  I  would  like  to 
look  at  the  human  assets  of  Uie  environment.  Obviously  learning  styles  are  an  important  aspect,  but 
1  would  suggest  that  Uie  roles  of  teachers  and  learners  arc  Uie  key  issues  not  learning  styles.  In  Uie 
limited  time  1  am  missing  out  Uie  more  global  institutional  and  organisational  factors  which  are  also 
key  factors.  1  shall  concentrate  on  Uie  learning  environment. 

In  Uie  learning  environment  learners  and  instructors  will  have  certain  expectations  of  Uieir  roles  from 
previous  experience.  So,  for  example,  learners  in  an  armed  services  environment  may  be  used  to  a 
passive  role,  wiUi  the  instructors  perceived  not  only  as  Uie  experts  but  also  Uie  controllers  of  discipline. 

I  would  like  to  explore  a  simple  idea  of  Uie  potential  roles  of  learners  and  instructors  as  Uiey  affect 
Uie  learning  environment. 

Learners  Roles 

The  focus  is  roles,  although  there  is  a  relationship  wiUi  learning  styles  and  approaches.  Learning  can 
occur  in  any  combination  of  skills,  cognitive  learning,  attitudes  and  cooperation  (ie  inler-pcrsonal 
skills).  I  would  like  to  look  at  four  roles  of  learners  (Harris  and  Bell,  1990) 

Receiver 

Detective 

Generator 

Facilitator 

The  learner  as  RECEIVER  may  be:  listening. to  somebody  talking  (it  could  be  Uie  instructor,  anoUicr 
learner),  watching  somebody  perfomiing  a  skill,  observing  Uie  behaviour  of  a  person  in  a  group.  The 
role  is  generally  a  silent  role  and  is  a  clear  expectation  in  a  didactic  learning  environment.  (It  has 
some  relationship  to  declarative  learning,  Rumelhart  and  Nonnan,  1981) 

The  learner  as  DETECTIVE  has  a  problem  to  solve,  the  solution  may  not  be  part  of  Uie  detective  role. 
That  problem  could  be:  how  to  enable  a  member  of  a  group  to  play  a  more  active  part  in  Uie  group, 
how  to  go  about  Uie  manipulation  of  a  piece  of  wood  to  bring  about  a  particular  shape  with  a 
particular  ^et  of  machines  and  tools,  how  to  solve  a  new  mathematical  problem.  The  role  is  a  more 
active  role,  where  there  is  an  expectation  for  the  learner  to  take  some  responsibility  for  Uie  learning. 
A  different  learning  environment  will  be  needed  to  enable  this  to  occur.  The  learner  has  to  be  aware 
of  the  different  expectations.  (It  is  related  to  procedural  learning,  Rumelhart  and  Nonnan,  1981). 
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The  learner  as  GENERATOR  is  trying  to  be  creative,  developing  new  ide/us.  new  relationships,  new 
skills.  It  is  a  role  wiih  even  more  responsibility.  A  role  often  excluded  in  many  courses  in  highc, 
education  where  credits  can  be  obtained  for  repealing  what  was  .sceived.  (it  is  associated  vvith 
generative  learning,  Wittrock,  1974). 

The  learner  as  FACILITATOR  is  probably  the  most  advanced  role  where  the  learner  is  abcui  enabling 
oilier  learners  to  succeed.  The  facilitation  can  occur  in  skills,  cognitive  learning,  it  can  occur  more 
obviously  in  cooperative  learning.  So  often  cooperative  learning  is  assumed  as  an  existing  skill  (for 
example  asking  learners  to  work  together  at  a  computer  terminal),  yet  the  role  of  learners  need  to  be 
changed  to  that  of  facilitator  to  enable  cooperative  learning  (see,  for  example,  Slavin,  1983). 

I  would  suggest  to  you  that  the  designer  not  only  has  to  be  aware  of  the  design  expectations  of 
learners'  roles,  but  also  what  learners  expectations  arc  in  the  context  of  learning.  If  the  learner  expects 
to  play  the  receiver  role,  then  designing  for  the  facilitator  role  has  potential  mismatches  with  the 
existing  learning  environment.  There  can  also  be  constraints  dictated  by  the  learning  environment:  the 
strategy,  materials,  equipment  may  necessitate  certain  roles. 


Instructors'  Roles 

Again  the  focus  is  on  roles,  this  time  for  Instructors. 
(Harris,  1979).   The  four  roles  are: 

Performer 


1  find  a  musical  analogue  useful  in  this  context 


Conductor 
Composer 
Critic 

Let  us  explore  these  roles  in  more  detail.  The  PERFORMER  is  the  didactic  presenter  (my  current 
role),  although  I  can  play  the  role  in  the  written  word,  at  a  piece  of  equipment  or  manipulating  a 
group!  However  the  last  is  perhaps  not  part  of  this  role! 

The  CONDUCTOR  role  is  the  instructor  who  is  helping  a  group  of  learners  who  may  all  be  doing 
different  learning  activities  at  the  same  time.  The  conductor  ensures  that  all  the  learners  have  help 
as  needed  and  also  takes  them  on  to  new  learning  activities  as  necessary.  The  learning  activities  can 
be  anything  from  maths  through  science  experiments  to  drama  rehearsals.  The  design  for  the  learning 
activity  is  very  dependent  on  the  role  of  the  instructor. 

The  COMPOSER  is  more  obvious  as  the  designer  of  the  learning,  usually  for  that  group  of  learners 
or  only  those  groups  with  whom  the  instructor  comes  into  contact.  I  suppose  the  ultimate  composer 
is  the  designer  of  curricular  resources  which  will  be  used  in  any  institutions  perhaps  across  states,  but 
I  have  restricted  the  role  to  those  who  are  mainly  instructors.  Obviously  if  there  is  a  learning 
environment  where  the  instructors  arc  used  to  the  role  of  composer  then  it  is  important  for  designers 
to  incorporate  some  part  of  this  role  in  the  design. 

The  CRITIC  role  is  that  of  the  evaluator  of  learners1  work,  of  the  learning  process. 
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Expectations 


Tnctc  arc  expectations  irom  learners  and  from  instructors  in  any  learning  environment.  Returning  to 
the  exampi?!  from  the  armed  services,  the  expectations  of  the  learners  and  of  the  instructors  would  be 
for  the  learners  lo  take  the  receiver  role  and  for  the  instructors  to  take  the  performer  role.  Any 
Seaming  design  which  changes  these  expectations  needs  careful  incorporation  of  enabling  the 
instructors  and  learners  to  take  up  new  roles.  It  is  insufficient  to  just  present  a  total  change  in  the 
expectations  of  roles.  It  will  be  necessary  to  provide  assistance  to  the  instructors  for  the  new  role  and 
also  for  the  learners  for  the  new  role.  Designers  need  to  incorporate  such  resources,  guidance  and 
assistance  to  enable  the  new  approach  to  be  taken  on  with  a  gradual  gain  in  confidence  rather  than  a 
step  function  change,  The  intention  of  any  design  is  to  enable  commiUnent  from  Uie  instructors  and 
the  learners  to  the  new  approach.  That  commitment  an  only  occur  when  there  is  a  deliberate  effort 
in  the  design  to  help  Uie  learners  and  Uie  instructors. 

The  Uirce  key  issues  (Tessmcr  and  Harris,  1992)  that  arise  arc 

Mismatches 

Attitudes 

Communicability 

The  idea  of  MISMATCHES  has  already  been  identified  in  Uie  previous  paragraph.  If  Uicre  arc 
mismatches  in  Uie  expectations  of  the  instructors  and  learners,  then  it  is  unlikely  that  Uie  design  will 
be  effective  in  the  learning  environment.  Mismatches  for  instructors  are  probably  more  influential  in 
providing  a  barrier  to  the  implementation  of  the  design.  Mismatches  with  learners  may  present 
ineffective  learning  unless  there  is  considerable  commiUnent  from  Uie  instructors. 

The  ATTITUDES  of  instructor  and  learners  arc  the  crucial  aspects  which  will  enable  or  prevent 
successful  outcomes  from  the  design.  The  designer  is  always  looking  for  commitment  from  instructors 
and  learners,  but  there  is  also  a  need  to  enable  the  commitment  by  careful  incorporation  of  persuasion 
and  development  during  Uie  actual  learning  process  (learning  for  both  instructor  and  Uie  learners  about 
Uie  changed  expectations). 

In  order  to  enable  Uie  change  in  attitudes  Uie  design  must  incorporate  COMMUN1CABILITY.  There 
is  litUe  point  in  having  a  carefully  designed  guide  for  instructors  which  puts  them  off  at  Uie  first 
stagc.many  of  us  will  be  familiar  with  early  (and  more  recent)  manuals  for  computer  software  which 
make  unreasonable  assumptions  about  the  knowledge  of  the  user.  It  is  essential  Uiat  there  is  a  clear 
communication  to  Uie  instructors  and  the  learners. 

Conclusion 

The  learning  environment  is  dependent  on  the  roles  of  Uie  learners  and  of  Uie  instructors.  The  design 
needs  to  take  account  of  Uie  current  expectations  of  Uie  learners  in  their  own  environment  (Harris  arid 
Tessmer,  1990).  If  there  is  to  be  a  change  in  role  for  Uie  instructors  and/or  Uie  learners  Uicn  Uie 
design  needs  to  enable  the  instructors  and  Uie  learners  to  become  accustomed  to  Uiose  roles  (You  may 
find  Uie  list  of  questions  in  Appcnd5x  A  helpful  during  design  processes).  Unless  Uierc  is  a 
commitment  to  the  approach  there  is  likely  to  be  a  poor  reception  for  Uie  new  approaches  and 
resources.  Not  only  does  Uicre  need  U>  be  a  commitment  from  Uie  instructors  and  Uie  learners  but  also 
from  Uie  organisation  and  the  institution  but  that  is  another  story!. 
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Appendix  A 


QUESTIONS  FOR  THE  DESIGNER  RELATING  TO  THE  ROLES  OF  THE 
INSTRUCTORS  AND  OF  THE  LEARNERS.  (Tessmer  &  Harris,  1992) 


What  arc  the  expectations  irom  uie  learners  in  uie  learning 

environment? 

How  are  they  expected  to  do  the  learning? 

When  are  they  expected  to  do  tne  learning  i 

What  responsibilities  arc  they  expected  to  take? 

Can  the  learning  environment  support  variations  in  Die  learners' 

role? 

Why  arc  the  learners  doing  me  learning.' 

What  do  tney  expect  to  icaro/ 

How  do  mey  expeci  10  icani: 

Where  do  mey  expeci  10  oo  mc  learning  ? 

wnai  responsi diii ues  ao  uicy  cajjcli  iu  uuvcj 

What  are  the  expectations  iro> 1  uie  inMiucior  in  uie  icaming 

environment? 

How  are  instructors  expected  to  neip  learners.' 

When  are  instructors  expected  to  neip  learners/ 

What  is  Uie  division  of  responsibility  between  the 

instructors  and  Uie  learners? 

Can  Uie  learning  environment  support  variations  in  Uie 

instructors'  role? 

What  expectations  do  instructors  have  from  their 

contract  of  employment? 

What  contact  dc  instructors  have  with  learners? 

What  responsibilities  are  instructors  expected  to  give  to 

learners? 

What  role  would  instructors  prefer  to  play? 

Hie  answers  to  these  questions  will  detemiine  the  design  of  Uie  learning.  By  using  explicit 
statements  of  these  expectations  the  designer  has  a  key  framework  as  a  basis  for  the  design. 
The  learners  may  wish  to  choose  between  alternatives  where  they  have  the  teaching  done  ON 
them,  the  teaching  done  WI  TH  them  or  the  teaching  done  BY  them.  If  the  learners  expect  to 
go  to  a  course  where  everything  is  done  for  litem  and  find  that  they  are  expected  to  work  in 
self  motivated  syndicates  negotiating  their  own  goals  etc,  they  may  feel  cheated  relative  to 
their  expectations.  To  prevent  this,  it  is  essential  that  the  designer  collects  information  to 
enable  the  decisions  to  meet  Uie  expectations  of  Uie  instructors  and  learners  or  to  identify  Uie 
needs  for  communication  or  persuasion  to  make  the  innovation  feasible.  To  collect  mis 
infonnation  Uie  designer  should  keep  the  following  principles  in  mind: 

•        Site  visits  are  essential. 

For  young  learners  talk  with  them  to  find  out  what  they  say  that  they  like. 


•  For  older  learners  use  group  techniques  (eg  snowball  technique). 

•  Fo-  instructors  the  snowball  technique  is  also  applicable. 

•  Suiyeys  of  learner  and  instructor  opinions  can  be  used  to  generalise  the  data 
from  the  group  techniques. 

Observation  of  the  current  practices  of  instructors  and  learners  will  provide 
further  evidence  and  ensure  triangulation. 
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Abstract 

Tiiis  paper  discusses  the  need  for  integrating  mental 
model  analysis  in  the  process  of  instructional  program 
development.  Investigations  into  the  effects  of  mental 
models  strategy  have  indicated  positive  effects  on 
learning  and  instruction.  It  was  proposed  that  in 
developing  instructional  materials  which  reguire 
complex  cognitive  process  in  learning,  the 
instructional  designer  identify  the  integrative 
goals/objectives  along  with  their  component  declarative 
and  procedural  knowledge,  and  develop  instructional 
strategies  that  can  help  learners  acquire  integrated 
outcomes  such  as  relevant  mental  models  in  the  specific 
domain.  The  procedures  and  examples  cc  the 
instructional  materials  development  with  mental  model 
analysis  are  illustrated. 
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Effects  of  Instructional  Design  with 
Mental  Model  Analysis  on  Learning 

The  concept  of  mencal  models  (Wilson  &  Rutherforu, 
1989;  Norman,   1986)  has  recently  been  introduced  to  the 
ifield  of  instructional  psychology  along  with  other 
constructs  of  knowledge  representation  such  as  schemata 
(Rumelhart,   1980),  frames  (Minsky,   1975),  or  scripts 
(Schank  &  Abelson,  1977),  Investigations  into  the 
effects  of  mental  models  strategy  in  relation  to 
learning  and  instruction  have  been  conducted  in  various 
domains  (Dyck  &  Mayer,   1989;  Gentner  &  Stevens,  1983; 
Mayer,   1989;  Kieras,   1987;  White  &  Frederiksen,  1986). 
Mayer,  Dyck,  and  Cook  (1984),   for  example,  investigated 
the  effects  of  mental  models  on  performance  in  causal 
systems.  They  provided  node  training  which  is  the 
conceptual  underpinnings  of  the  key  definitions 
involved  in  the  passage,  and  link  training  provided  by 
passages  that  emphasized  the  main  relations  among 
concepts •  They  predicted  that  subjects  who  were 
provided  with  the  mental  models  aids  would  build  a 
coherent  mental  model  of  the  system.  Compared  to  the 
control  group,  the  mental  model  group  significantly 
recalled  more  information  concerning  the  main  concepts 
and  their  relationships,  and  performed  better  in 
creative  problem  solving . 

In  a  study  by  Kieras  and  Bovair  (1984),  subjects 
who  knew  how  a  control  device  worked  could  learn  and 
infer  how  to  operate  it  much  more  efficiently  than 
subjects  who  did  not  have  this  information.   In  their 
experiment,   the  mental  model  group,  who  were  exposed  to 
the  device  model  that  describes  the  internal  mechanism 
of  the  device  before  receiving  the  procedure  training 
on  how  to  operate  a  control  device,   learned  the 
procedures  faster,  retained  the  procedures  more 
accurately,  and  executed  the  procedures  faster  than  the 
rote  group.  These  results  support  instructional 
strategies  that  help  learners  build  mental  models, 
which  will,   in  turn,   promote  subsequent  learning  and 
problem  solving. 

Streitz  (1988)  distinguished  a  mental  model  from  a 
conceptual  model  in  that  a  mental  model  is  a  subjective 
knowledge  representation  that  is  an  idiosyncratic  and 
very  personal  model,  while  a  conceptual  model  is  a 
model  developed  by  scientists  or  designers.   In  studies 
of  instructional  applications  of  mental  models,  Mayer 
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(1989)  concluded  that  by  using  conceptual  models,  which 
are  words  and/or  diagrams  that  are  intended  to  help 
learners  build  mental  models  of  the  system  being 
studied,   learners  could  improve  their  recall  of 
conceptual  information  and  increase  their  creative 
solutions  on  transfer  problems 

In  the  present  study,  the  author  reserves  the  term 
mental  model  for  a  person's  internal,  domain-specific 
representation  which  may  be  incomplete  or  unstable,  and 
the  term  relevant  mental  model  for  an  internal,  domain- 
specific  representation  that  is  relevant  and  useful  for 
a  person's  subsequent  understanding  and  in  problem 
solving  in  the  domain . 

This  paper  presents  a  model  for  systematic 
instructional  design  that  includes  mental  model 
analysis,  and  the  procedures  used  in  developing 
computer-based  instructional  materials  in  the  area  of 
statistical  hypothesis  testing.  The  instructional 
design  model  is  based  on  the  premise  that  the  objective 
of  learning  is  to  achieve  expert-like  mental  models, 
and  instruction  should  be  designed  to  help  learners 
build  relevant  mental  models  in  the  specific  domain.  In 
addition,  the  instructional  effects  which  resulted  from 
the  application  of  the  mental  model  strategies  in  the 
domain  of  introductory  hypothesis  testing  are 
discussed . 


A  Model  for  Instructional  Systems  Design 
with  Mental  Model  Analysis 

The  quality  of  an  instructional  system  would  be 
considered  high  when  adequate  contents  are  presented  to 
learners  using  proper  instructional  strategies .  In  a 
recent  discussion  on  integrative  goals  for 
instructional  design,  Gagne  and  Merrill  (1990) 
suggested  the  integration  of  multiple  objectives  for 
comprehensive  purposeful  activities.  They  considered 
integrated  objectives  as  an  enterprise,   and  proposed 
that  integrative  goals  be  represented  in  cognitive 
space  by  enterprise  schemas  (similar  to  the  concept  of 
mental  models)  whose  focal  integrating  concept  is  the 
integrative  goals. 

As  Gagne  and  Merrill  (1990)  suggested  that 
instructional  design  must  specify  the  conditions  for 
acquisition  of  enterprise  schemas  or  mental  models,  the 
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content/task  analysis  which  emphasizes  on  mental  models 
are  expected  to  lead  to  considerable  changes  in 
instructional  strategies  and  material  development. 
Thus,   the  author  proposes  that  mental  model  analysis  be 
integrated  as  one  of  the  phases  in  the  systematic 
design  of  instructional  materials  so  that  cognitive 
task  analysis  can  produce  instructional  strategies, 
which  provide  conditions  for  learners  to  build  relevant 
mental  models  in  the  domain. 

Phases  of  Systematic  Design  of  Instructional  Materials 

The  model  described  in  this  paper  is  categorized 
into  three  phases  in  a  systematic  design  of 
instruction:    (1)  analysis  of  instructional  outcomes, 
(2)  development  of  instructional  material,   and  (3) 
implementation,   evaluation,  and  revision  of  the 
instructional  material. 

Phase  I:  Analysis  of  instructional  outcomes .  I n 
this  phase,   information  about  the  resulting  outcomes 
are  to  be  provided.  After  the  overall  instructional 
goals  are  identified,  relevant  mental  models  for  the 
domain  will  be  determined  through  cognitive  task 
analysis  (i.e.,  mental  model  analysis).  The  mental 
model  analysis  identifies  declarative  and  procedural 
knowledge  involved  in  the  domain,  and  determines 
whether  mental  model  strategies  would  be  helpful  in  the 
particular  domain.  The  end  results  of  this  phase  would 
be  the  determination  of  the  relevant  mental  models  for 
teaching  learners . 

Phase  II :  Development  of  instructional  material . 
To  ensure  optimum  learning,   learners1  entry  level 
knowledge  and  skills  are  first  identified  in  this 
phase.  Using  the  information  on  the  learners1  entry 
level  and  the  relevant  mental  models  for  teaching 
novices,   instructional  strategies  to  help  learners 
build  relevant  mental  models  of  the  domain  will  be 
developed.  Employing  both  macro  and  micro  levels  of 
instructional  design  strategies  (Reigeluth,   1983),  as 
well  as  the  mental  model  strategies  developed  in  this 
phase,   instructional  materials  are  developed  along  with 
the  selection  of  appropriate  media  to  deliver  the 
instruction.  Methods  for  assessing  the  learners1 
performance  are  also  developed.   In  other  words,  whether 
learners  acquired  relevant  mental  models  or  not  should 
be  measured  through  appropriate  assessment  procedures. 
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Phase  III:   Implementation,  evaluation,  and 
revision  of  the  instructional  material .  In  this  phase, 
the  instructional  materials  developed  in  phase  II  are 
implemented  for  evaluation.  The  effectiveness  and 
practical  feasibility  of  the  instruction  are  to  be 
revealed  through  the  formative  evaluations  (Dick  & 
Carey,   1985),  and  necessary  revisions  should  be  made 
according  to  evidence  from  the  evaluation.  In  the 
following  section,   each  step  of  the  model  for  the 
systematic  design  of  instructional  material  is 
described . 

Procedures  for  Developing  Instructional  Materials 

Figure  1  presents  a  model  of  systematic  approach 
to  instructional  materials  development.  In  this  paper, 
the  emphasis  is  given  on  the  application  of  mental 
model  analysis  to  each  step  in  describing  the 
procedures  for  instructional  materials  development. 


Insert  Figure  1  about  here 


1 .  Identify  instructional  goals/objectives 

Goals  may  be  stated  as  the  intended  outcomes  of 
instruction.   In  specifying  learning  outcomes,  Gagne, 
Briggs,   &  Wager  (1988)  suggested  that  assigning 
learning  objectives  to  five  major  categories  of  human 
capabilities  (i.e.,   intellectual  skills,  cognitive 
strategies,  verbal  information,  motor  skills,  and 
attitudes)  can  simplify  the  instructional  planning. 
However,   in  situation^  where  complex  cognitive 
processes  are  involved  in  learning,   that  is,  when 
instruction  involves  more  than  simple  rules/procedures 
or  facts,   instructional  objectives  may  be  specified 
comprehensively  so  that  the  following  content/task 
analysis  can  be  conducted  to  provide  learning 
conditions  for  comprehensive  activities .  For  instance, 
in  developing  instructional  materials  for  teaching 
introductory  hypothesis  testing,  an  example  of 
instructional  objectives  would  be:  The  students  will  be 
able  to  make  inferences  about  the  nature  of  the 
population  when  given  problems  in  hypothesis  testing, 
through  specifying  null  and  alternative  hypotheses, 
setting  the  region  of  rejection,  computing  the  test 
statistic,  making  decisions  about  the  hypotheses,  and 
drawing  valid  interpretations  on  the  decision  made. 
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This  example  of  an  objective  indicates  that  to  achieve 
a  higher  level  of  performance  in  problem  solving  in 
this  subject  matter,  the  students  should  have  both 
conceptual  and  procedural  knowledge  involved  in 
hypothesis  testing,  and  apply  or  synthesize  that 
knowledge  when  they  encounter  problem-solving 
situations . 

Instructional  designers,  then,  may  consider  the 
objectives  as  coherent  cognitive  structures  of 
integrated  knowledge/skills  of  the  to-be-learned 
materials,  ^o  that  the  following  steps  in  the  design  of 
instruction  would  focus  on  the  learners'  outcomes  which 
are  goal-related  knowledge  and  skills,  or  stated 
differently,  coherent  and  relevant  mental  models  of  the 
particular  domain. 

Then,  what  and  how  should  the  specific  knowledge 
or  skills  be  presented  to  help  learners  develop  the 
relevant  mental  models?  This  question  calls  for  the 
analysis  of  mental  models  in  the  domain,  which  is  the 
topic  of  the  next  section. 

2.  Conduct  mental  model  analysis. 

The  purpose  of  mental  model  analysis  is  to 
determine  relevant  mental  models  for  teaching  learners 
in  accordance  with  the  instructional  objectives 
specified  in  step  1.  By  examining  both  experts'  and 
less-than-experts 1  mental  models,   relevant  mental 
models  learners  should  acquire  as  well  as  learners1 
common  misconceptions  in  the  domain  will  be  determined. 
Figure  2  presents  the  procedure  involved  in  conducting 
mental  model  analysis . 


Insert  Figure  2  about  here 


' % stermining  if  the  mental  model  strategy  can  be 
heJ ;   al .  To  determine  if  the  mental  model  strategy 
would  help  students  learn  materials  in  a  particular 
domain,  available  texts  or  documents  would  first  be 
analyzed  to  specify  what  knowledge/skills  should  be 
taught.  In  addition,  subject  matter  experts  may  play  a 
significant  role  in  content/task  analysis.  Through  the 
content  analysis,  declarative  and  procedural  knowledge 
(i.e.,  facts,  concepts,  procedures,  rules,  or 
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principles)  involved  in  the  domain  are  identified.  The 
result  of  content  analysis  may  reveal  that  the 
particular  domain  to  teach  involves,   for  example, 
concepts  and  principles,  or  it  may  involve  only  facts 
and  simple  procedures. 

Upon  the  identification  of  knowledge/skills  to  be 
taught,   it  may  be  decided  whether  mental  model  strategy 
is  helpful  in  teaching  the  particular  domain.  Kieras 
(1988)  identified  the  situations  when  mental  models  are 
not  useful  in  the  domain  of  device  maintenance:    (1)  if 
the  procedures  are  easy  to  learn  by  rote;    (2)  if  the 
device  is  so  simple  that  the  learner  does  not  need  to 
make  inferences;    (3)  if  mmtal  models  are  too  difficult 
or  complicated  for  the  learner  to  acquire  and  use;  and 
(4)  if  mental  models  fail  to  support  the  inferences 
that  the  learner  needs  Lo  make,  or  alternatively, 
support  inferences  that  the  learner  does  not  need  to 
make.  In  his  review  on  the  effects  of  conceptual  models 
that  are  intended  to  help  learners  build  mental  models, 
Mayer  (1989)  focused  on  reviewing  explanative  material. 
Explanative  material  allows  learners  to  build  and  use 
models  that  explain  the  information.  Mayer  (1989)  added 
that  the  reason  he  focused  on  explanative  material  in 
his  review  was  because  meaningful  methods  of 
instruction  can  only  have  an  effect  for  learning  of 
material  that  is  potentially  meaningful. 

Thus,   to  determine  if  ment  ,1  model  strategy  is 
helpful,   it  should  be  analyzed  to  find  out  if  the 
content  requires  meaningful  learning  or  inferencing 
process  based  on  knowledge  acquired  from  the 
instruction.  The  designer  may  analyze  if  the  mental 
model  strategy  would  help  learners  in  selecting 
information  to  pay  attention  to,   in  organizing  incoming 
information  in  short-term  memory,   in  integrating  prior 
knowledge  from  long-term  memory  with  incoming 
information,  and  in  encoding  the  resultant  learning 
outcome  in  long-term  memory  (Mayer,  1989). 

In  the  domain  of  introductory  hypothesis  testing, 
existing  textbooks  were  first  analyzed.  The  declarative 
and  procedural  knowledge  involved  in  hypothesis  testing 
were  identified,  which  were  concepts,  principles ,  and 
procedures  involved  in  hypothesis  testing.  At  thxs 
stage,  it  seemed  that  there  could  be  a  small  set  of 
mental  models  which  could  be  either  relevant  or 
incorrect  after  students  are  exposed  to  problems  on 
hypothesis  testing . 
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Relevant  mental  models  to  teach.  To  determine 
relevant  mental  models  to  teach  in  introductory 
hypothesis  testing,  a  pilot  study  was  conducted.  The 
procedures  included:   (1)  experts'  and  intermediates1 
mental  models  were  elicited;   (2)  the  elicited  mental 
models  were  compared  with  the  textbook  contents 
analyzed;  and  (3)  the  relevant  mental  model  to  teach 
novices  were  determined.  This  pilot  study  is  described 
in  detail  elsewhere  (Hong,  1990). 

The  results  of  the  pilot  study  revealed  that 
experts1  verbal  protocols  on  the  problem  solving 
process  were  incomplete  such  that  their  protocol 
indicates  combined  inference  steps .  This  is  largely 
because  their  knowledge  were  compiled  through  their 
extended  learning  process  and  extensive  practice  in 
problem-solving  activities.  However,   in  considering  the 
learnability  of  expert  models,  several  additional 
questions  were  presented  to  experts  in  order  to 
decompile  their  knowledge,  such  as,   "if  you  teach 
students  to  solve  this  problem,  what  would  you  do?M  In 
investigating  the  intermediates'  mental  models,  it  was 
found  that  their  mental  models  were  especially  useful 
in  designing  instruction  because  their  knowledge  was 
not  compiled  and  their  common  misconceptions  were 
revealed. 

The  major  difference  between  experts  and 
intermediates  in  the  domain  of  introductory  hypothesis 
testing  was  their  conceptual  understanding:  While  all 
experts  had  a  solid  conceptual  understanding,  many 
intermediates  did  not.  However,  the  diagrammatic 
problem  representation  was  useful  for  most  subjects1 
problem  solving,  especially  for  intermediates. 
Considering  the  importance  of  conceptual  understanding 
and  usefulness  of  diagrammatic  representation,  the 
mental  model  to  teach  novices  in  the  context  of 
introductory  hypothesis  testing  was  defined  as  concepts 
and  rules  involved  in  hypothesis  testing  in 
diagrammatic  representation . 

Research  on  text  illustration  supports  the  use  of 
diagrammatic  representation  in  teaching  or  learning  the 
instructional  materials  and  solving  problems:  Students 
who  received  instruction  such  as  text  passages  which 
include  illustrations  retained  more  information  than 
those  who  received  instruction  without  illustrations 
( Alesandrini,   1984;  Anglin  &  Stevens,   1986;  Curtis, 
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1988;  Reid  &  Beveridge,   1986;  Rusted  &  Hodgson,  1985). 
Studies  on  topologies  of  device  models  (de  Kleer  & 
Brown,   1983;  Kieras,   1984),  which  used  diagrammatic 
representations  of  tiie  models,  also  show  the  effects  of 
the  diagrammatic  presentation  of  the  instruction* 

3.  Identify  the  learners1   entry  level  knowledge/skills. 

The  most  common  method  for  assessing  prerequisite 
knowledge  and  skills  may  be  administering  a  pretest  to 
students.  The  test  may  contain  items  to  assess  the 
knowledge  and  skills  students  should  have  when  the 
instructor  begins  to  teach  new  information  (Dick  & 
Reiser,   1989) . 

In  the  present  hypothesis  testing  study, 
prerequisite  knowledge/skills  that  a  student  should 
have  were  identified,  and  it  was  decided  that  a 
prerequisite  learning  and  test  session  be  provided.  The 
intention  underlying  the  test  item  development  was  that 
the  subjects  learn  the  prerequisites  by  solving 
problems  and  reading  the  informative  feedback.  The 
subjects  were  given  the  opportunities  of  answering 
twice  in  case  they  made  incorrect  responses. 
Informative  feedback  was  given  whether  the  response  was 
correct  or  not . 

4.  Develop  instructional  strategies  considering  mental 
models . 

To  determine  the  instructional  strategies  to  help 
learners  build  relevant  mental  models  in  hypothesis 
testing,   the  investigator  designed  four  instructional 
units  about  the  hypothesis  testing  for  one-sample  case 
for  the  mean  (variance  known).  The  four  instructional 
units  basically  contained  the  same  information,  but 
were  designed  in  different  presentation  sequences 
(separate  and  combined)  and  presentation  modes 
(diagrammatic  and  descriptive).  Briefly,   the  sequencing 
of  the  two  instructional  units,  separate-diagrammatic 
and  separate-descriptive  units,  were  the  same  in  that 
concepts  involved  in  hypothesis  testing  were  presented 
first,  and  then  followed  by  procedural  and  quantitative 
instruction.  For  the  combined-diagrammatic  and 
combined-descriptive  units,  the  concepts  and  procedures 
were  presented  simultaneously. 

The  primary  difference  between  the  diagrammatic 
and  descriptive  presentation  modes  was  the  frequency  of 
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the  diagrams  presented  in  the  instructional  units.  In 
the  diagrammatic  presentation  mode,   18  diagrams  were 
presented,  while  six  diagrams  were  presented  for 
descriptive  mode  (see  Hong  &  O'Neil,   in  press,  for 
detail ) . 

The  rationale  underlying  the  development  of  the 
four  units  were  from  the  mental  model  analysis 
conducted  in  the  earlier  stage  and  the  mental  model 
literatures.  Discussing  the  progressions  of  qualitative 
models  in  circuit  behavior,  White  and  Frederiksen 
(1986)  advocated  that  qualitative  conception  of  the 
domain  should  be  acquired  before  quantitative  models  be 
introduced  so  that  the  causal  relations  in  the  domain 
could  be  obvious.  They  argue  for  the  importance  of 
presenting,   in  the  initial  stages  of  learning, 
qualitative,   causally  consistent  models  so  that 
students  can  gain  an  understanding  of  basic  concepts 
and  principles. 

In  their  study  on  transfer  of  computer  language 
comprehensive  skill,  Dyck  and  Mayer  (1989)  conducted  a 
cognitive  analysis  of  BASIC  languages.  Then  they 
provided  sequential  and  simultaneous  instructional 
methods .   Iii  the  simultaneous  method,  the  semantics  of  a 
language  were  taught  within  the  syntax  of  the  language, 
and  in  the  sequential  method,  the  semantics  of  a 
language  were  taught  prior  to  the  syntax  of  the 
language  so  that  learning  of  semantic  of  a  language  can 
influence  learning  of  the  other  language.  Results 
supported  the  use  of  a  sequential  method  of  programming 
language  instruction,  because  subjects  who  had  previous 
experience  in  the  learning  of  English  procedural 
language  (semantics)  learned  BASIC  faster  and  more 
accurately . 

These  studies  illustrates  thut  the  teaching  of 
concepts  prior  to  procedural/quantitative  instruction 
may  help  students  build  relevant  mental  models.  In 
addition,   as  discussed  earlier,   the  provision  of 
instruction  in  diagrammatic  forms  rather  than 
descriptive  forms  would  facilitate  understanding  and 
solving  problems* 

5.  Develop  instructional  materials  and  tests. 

At  this  stage  the  actual  instructional  messages , 
such  as  directions,   information  presentations, 
practices,   feedback,   and  test  items  were  written  on 
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paper  in  draft  form.  In  the  selection  of  media,   it  was 
decided  that  the  prerequisite  test  and  instructional 
units  be  presented  by  computer-based  instruction  (CBI). 
The  rationale  for  using  CBI  in  statistics  was  two  fold: 
(1)  evidence  from  the  literature  indicates  that  CBI  is 
useful  in  statistics  instruction.  For  example, 
Varnhagen  and  Zumbo  (1990)  found  that  computer-assisted 
instruction  had  a  significant  effect  on  student 
attitude  toward  statistics  instruction  and  had  an 
indirect  effect  over  performance  through  its  influence 
on  affect.  They  recommended  that  technical  courses  such 
as  introductory  statistics  should  ideally  be 
supplemented  with  CBI  in  order  to  provide  optimal 
learning  experiences;   (2)  the  use  of  CBI  facilitates 
the  research  issues,  e.g.,  software  ensures  that  the 
experimental  treatments  were  provided  as  designed.  Upon 
completion  of  written  draft  of  the  material, 
storyboards  were  produced  with  each  page  corresponding 
to  a  separate  computer  screen  display.  The  computer- 
based  instructional  materials  were  programmed  in  an 
authoring  system,  IconAuthor  (AIMtech,   1989)  and  were 
delivered  by  an  IBM  personal  computer. 

After  the  prerequisite  test  and  the  four 
instructional  units  were  developed,   they  werj  checked 
to  ensure  that  the  instructions  were  coherent  and  the 
program  was  free  of  functional  errors  before  the 
formative  evaluation  took  place.  The  following  issues 
were  considered  when  the  investigator  worked  through 
the  program  (Alessi  &  Trollip,   1985;  Olson  &  Wilson, 
1985) : 

A.  Is  the  use  of  words  and  symbols  involved  in  hypothesis  testing  consistent 
throughout  the  program? 

B.  Is  the  spelling,  grammar,  punctuation,  and  spacing  free  of  error  and 
consistent? 

C.  Are  the  directions  properly  and  clearly  given  for  the  subjects  to  proceed  with 
the  experiment? 

D.  Is  the  information  covered  in  the  treatment  complete  and  accurate,  and  placed 
in  the  proper  sequence?  And,  is  the  design  method  used  in  the  study  well 
reflected  in  the  information  provided? 

E.  Are  the  questions  and  feedback  handled  properly? 

1)  Are  the  practice  items  placed  in  the  proper  sequence?  In  other  words, 
are  the  practice  items  related  to  the  information  being  presented? 

2)  Are  the  questions/practice  items  unambiguous? 

3)  Is  the  feedback  appropriate  and  increasingly  informative  after  each 
successive  incorrect  response? 

4)  Are  the  subjects*  performance,  i.e.,  scores,  number  of  attempts,  and 
time  to  completion,  recorded  accurately? 
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F.  Is  the  screen  designed  properly? 

1)  Is  the  amount  of  information  proper  for  each  screen? 

2)  Are  the  diagrams  drawn  in  the  proper  place? 

3)  Is  the  type  size  and  style,  highlights,  inverse,  or  animation  used  in  an 
instructional  relevant  way,  but  not  overused? 

4)  Are  the  feedback  messages  displayed  on  the  lower  part  of  the  screen  in 
a  consistent  manner? 

5)  Is  the  screen  continued  from  tne  last  screen  noted  by  the  heading  and 
continuation  remark? 

6)  Is  the  timing  of  text  output  relevant  or  controlled  by  the  subjects? 

Posttest  items  were  prepared  in  paper-and-pencil 
format  because  the  mode  of  the  posttest  was  think-aloud 
protocol.  The  posttest  consisted  of  17  problems 
representing  major  concepts  or  rules  involved  in 
introductory  hypothesis  testing.  Each  problem  was  typed 
on  a  separate  sheet  and  subjects  could  use  the  blank 
space  for  computation  or  drawing  diagrams  while 
performing  the  think-aloud  protocol. 

Coding  system  for  the  protocol  analyses  was 
developed  based  on  the  posttest  items  and  a  theoretical 
framework  of  relevant  mental  models  in  hypothesis 
testing.   In  developing  the  coding  categories,  no 
attempt  was  made  to  deal  with  all  the  information  in 
the  think-aloud  protocols.   Instead,  only  selected 
features  of  the  protocols  related  to  the  task  were 
included  in  the  categories.  Since  the  present  study  was 
aimed  at  finding  if  the  subjects  built  relevant  mental 
models  in  hypothesis  testing,  a  coding  system 
describing  the  information  processing  leading  to  the 
solution  was  developed.  The  categories  in  the  system 
also  included  misconceptions  found  from  the  pilot  study 
and  formative  evaluation.  The  examples  of  the  coding 
system  and  scoring  procedures  can  be  found  in  Hong  and 
O'Neil  (in  press) 

6 .  Formative  evaluation . 

After  the  instructional  program  development  was 
completed  and  steps  were  taken  to  ensure  the  program 
was  free  of  functional  errors,  a  two-stage  formative 
evaluation  was  conducted:  one-on-one  testing  and  small- 
group  testing  (Dick  &  Carey,  1985). 

One-on-one  testing.  The  purpose  of  this  testing 
was  to  revise  the  instructional  program  while  it  was 
being  developed  by  using  students  similar  to  those  for 
whom  the  lerson  was  designed.  Because  of  the  small 
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numbers  involved  in  this  testing,   the  information 
gathered  from  the  testing  could  only  be  used  to  aid 
further  revision,  but  not  for  a  definitive  statement  of 
fact  (Alessi  &  Trollip,   1985).  Materials  used  in  this 
testing  were  CBI  instructional  units  developed  by  the 
investigator. 

While  each  subject  worked  through  the  program,  the 
investigator  attended  the  whole  session  and  took  notes 
through  unobtrusive  observation.  The  investigator  asked 
the  subjects  to  make  comments  about  the  program 
whenever  they  encountered  difficulty  in  understanding 
the  directions,  presentation,  and/or  questions.  As  soon 
as  the  subjects  finished  the  t-eatment  session,  the 
investigator  discussed  with  the  subjects  the  comments 
they  made  and  observations  the  investigator  wrote  down. 

For  the  posttesu,  think -aloud  protocol  method  was 
employed  and  recorded  for  the  entire  session.  After  the 
test,   the  subjects  were  asked  what  they  thought  of  the 
whole  session.  Findings  from  the  one-on-one  test  were 
used  when  revisions  were  made  (see  Hong,  1990) 

Small  group  testing.  After  the  one-on-one  testing, 
the  instructional  program  including  prerequisite  test, 
four  treatment  units,  and  posttest  items  were  revised. 
In  this  stage  of  formative  evaluation,   it  was  intended 
to  test  the  revised  instructional  program  and  further 
revise  it  before  the  main  study.  Even  though  the  title- 
-small  group  testing--was  borrowed  from  Dick  and  Carey 
(1985),   the  testing  was  conducted  at  one  to  one  level 
because  think-aloud  protocol  was  employed  for  the 
pos'.test.  As    with  the  one-on-one  testing,  because  of 
the  small  number  of  subjects  were  involved,  the 
information  from  this  testing  was  used  only  to  guide 
revision     .iot  for  any  kind  of  statistical  testing. 

In  this  testing,   the  subjects  were  not  allowed  to 
ask  any  questions  while  studying  the  material  except 
with  the  prerequisite  test.  However,   they  were 
encouraged  to  make  notes  of  comments  during  the 
sessions.  The  investigator  sat  with  the  subjects  while 
they  studied  through  the  program  and  took  notes  of 
their  behavior.  A  tape  recorder  was  used  throughout  the 
posttest  session  and  the  subjects  were  encouraged  to 
think  out  loud.  Findings  from  the  small  group  testing 
were  used  for  revision. 
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7.  Revision  of  instructional  program* 

Data  collected  ,7rom  pilot  testing,  one-on-one,  and 
small-group  testing  were  used  for  revising 
instructional  program.   In  brief,  some  of  test  items 
were  revised  or  deleted;  feedback  information  was 
elaborated;  help  screens  were  added;  diagrams  were 
elaborated  and  explained;  and  coding  categories  were 
revised  by  analyzing  each  subject1 s  protocols. 


Effects  of  Mental  Model  Strategy 
on  Instruction  and  Learning 

To  test  if  the  mental  model  strategies  (i.e., 
separate  and  diagrammatic  presentation)  had  a  positive 
effect,   subjects  wero  drawn  from  students  who  were 
taking  introductory  statistic  courses.  The  subjects  did 
not  have  knowledge  on  the  introductory  hypothesis 
testing  at  the  time  of  the  experiment,  but  had  learned 
the  prerequisite  information.  That  is,  the  experiment 
was  conducted  just  before  the  subjects  learn  the 
hypothesis  testing  in  their  classes.  The  subjects  were 
grouped  into  blocks  according  to  their  educational 
levels,   i.e.,   27  graduates  and  29  undergraduates,  and  a 
random  assignment  to  each  treatment  combination  was 
carried  out  separately  for  each  block  of  subjects. 

An  analysis  of  covariance  with  a  randomized  block 
factorial  design  was  performed  on  subjects1  problem- 
solving  scores.  The  prerequisite  test  score  was  used  as 
a  covariate  in  the  data  analysis.  In  addition,  an 
analysis  of  variance  waL  conducted  to  examine  whether 
the  frequencies  of  using  diagrammatic  problem 
representation  differed  by  the  differential  treatments 
of  presentation  mode  and  presentation  sequence.  Data 
analyses  of  the  study  are  described  in  full  detail 
elsewhere  (Hong  &  O'Neil,   in  press). 

The  results  of  the  study  provide  evidence 
concerning  the  instructional  strategies  which  help 
learners  build  relevant  mental  models  in  introductory 
hypothesis  testing.  First,  providing  conceptual 
instruction  prior  to  procedural  instruction 
significantly  facilitated  understanding  the  concepts 
and  the  procedures  involved  in  hypothesis  testing. 
Second,   instruction  using  extensive  diagrammatic 
representation  facilitated  subjects1  development  of 
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representational  ability  for  understanding  the 
instruction  by  building  diagrammatic  mental  models. 

Research  on  mental  models  has  found  that  the  use 
of  mental  model  strategy  can  enhance  learners' 
acquisition  of  knowledge,  especially  where  inference 
process  is  necessary  (Kieras,   1988;  Mayer,   1989).  The 
present  study  suggests  that,   in  the  course  of 
instructional  design  and  development,  mental  model 
analysis  be  conducted  to  determine  appropriate 
instructional  strategies . 
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Mental  Model  Analys 
Figure  Caption 

Figure  1,  A  model  for  instructional  systems  design  with  mental  models 
analysis. 

Figure  2.  Procedures  for  conducting  mental  model  analysis. 
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Abstract 

This  research  attempted  to  replicate  ai   extend  results  from  a  previous  study  examining  the  effects  on 
achievement  of  generating  summaries  or  analogies  while  reading  a  lengthy  text  Before  the  study,  1 1 1 
students  were  classified  as  high  or  low  ability  and  were  randomly  assigned  to  paired  or  individual 
conditions  and  to  one  of  three  experimental  treatments:  a  summaries  group;  an  analogies  group;  or,  a 
control  group.  Posttest  achievement  was  higher  for  students  who  generated  summaries  (mean  =21.41) 
than  for  students  who  generated  analogies  (mean  =  17.75).  Furthermore,  a  significant  interaction 
indicated  that  high  ability  students  learned  more  effectively  studying  alone  (mean  =  25.47)  than  in  pairs 
(mean  =  18.55).  Reasons  for  the  ineffectiveness  of  the  paired  treatment  are  discussed. 

Generative  Learning  in  Small  Groups 
Students  often  read  instructional  texts  superficially  resulting  in  poor  recall  and  transfer. 
Generative  learning  attempts  to  improve  learning  from  text  by  providing  students  with  methods 
that  stimulate  deeper  processing  (Wittrock,  1990).  Generative  learning  emphasizes  forming 
relations  among  information  to  be  learned  and  between  the  information  and  the  learner's  knowledge 
and  experiences. 

In  a  recent  study,  Wittrock  and  Alesandrini  (1990)  examined  the  effects  of  three  processing 
strategies  on  students1  ability  to  comprehend  and  retain  information  from  text  After  reading  each 
paragraph  of  a  text,  students  employed  one  of  two  generative  learning  techniques  (summarizing 
each  paragraph's  main  idea  or  writing  an  analogy  associating  the  text  with  the  reader's  personal 
knowledge  base)  or  simply  re-read  each  paragraph.  Students  who  generated  analogies  or  summaries 
demonstrated  higher  posttest  achievement  than  did  students  who  re-read  the  text,  but  no  differences 
were  found  between  the  generative  treatments. 

Although  achievement  differences  for  the  two  generative  activities  were  not  different,  data 
indicate  that  students  were  more  facile  generating  summaries  than  analogies.  During  the 
treatments,  students  generated  appropriate  summaries  with  99%  success  and  analogies  with  75% 
success.  These  data  suggest  that  generating  analogies  is  more  difficult  than  generating  summaries. 
Moreover,  difficulties  generating  analogies  may  have  suppressed  students'  posttest  achievement. 
Learning  for  the  analogies  treatment  may  have  improved  if  students  had  generated  more  effective 
analogies. 

Content  organizing  and  integrating  activities  may  be  more  effective  when  working  with  a 
partner  rather  than  alone.  Although  studies  have  focussed  on  learners  working  individually, 
generative  learning  may  be  more  effective  in  groups.  The  cognitive  effects  of  working  in  small 
groups  are  well  established.  A  meta-analysis  comparing  group  with  individual  learning  produced 
an  effect  size  of  .63  favoring  grouping  (Johnson  &  Johnson,  1989).  Furthermore,  in  a  study 
comparing  the  effects  of  summarizing  text  individually  or  in  groups,  students  v  ho  generated 
summaries  in  dyads  learned  more  than  students  who  generated  summaries  alone  (O'Donncll  et 
al.,1987).    Students  often  leam  more  effectively  in  an  environment  that  provides  opportunities  to 
share  information  and  engage  in  constructive  conflict 

Generating  analogies  in  small  groups  may  also  stimulate  learning.  Difficult  tasks  arc 
often  solved  more  effectively  in  small  groups  than  when  working  alone  (King,  1989).  However, 
two  important  questions  exist  concerning  the  formation  of  a  group  analogy.  First,  can 
collaborating  students  create  more  effective  analogies  than  students  working  alone?  Second,  can  an 
analogy  formed  by  consensus  improve  learning?  Cooperation  may  deepen  processing  and  improve 
recall  for  both  partners.  Alternatively,  a  group  analogy  may  furnish  information  that  benefits  only 
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one  partner  or  is  not  particularly  useful  to  cither  group  member  (Danscrcau,  1987).  In  such  cases, 
group  learning  may  be  no  more,  and  potentially  less,  effective  than  studying  alone. 

The  purposes  of  the  present  study  were  to  replicate  and  extend  Wittrock  and  Alcsandrini's 
(1990)  study.  We  investigated  the  effects  of  using  generative  learning  techniques  to  learn  from  text 
while  studying  alone  and  in  small  groups.  Furthermore,  we  examined  whether  students  working  in 
groups  generated  more  effective  summaries  and  analogies  during  instruction  than  did  students 
working  alone. 

Method 

SllfcifiClS 

A  sample  of  M 1  undergraduate  students  from  a  mid-western  university  completed  the  study. 
Students  received  er  /a  course  credit  in  ;omputer-scicnce  for  participating. 
Materials 

E[£l£StS.  The  purpose  of  the  pretest  was  to  measure  verbal-analytic  ability.  Verbal-analytic 

ability  was  measured  with  the  Similarities  Subtest  of  the  Weshsler  Adult  Intelligence  Scale1  (WAIS) 
(Wechsler^  1958).  Si  udents  were  classified  as  high  or  low  ability  according  to  pretest  performance. 
High  ability  students  scored  above  the  mean  and  lew  ability  students  scored  below  the  mean  on  the 
pretest.  The  mean  score  for  the  high  ability  students  on  the  pretest  was  the  73rd  percentile  and  mean 
for  the  low  ability  student  was  the  53rd  percentile. 

Tent.  Subj  cts  read  a  6,200  word  chapter  about  marine  life^  The  text,  which  was  the  same 
as  that  used  in  several  previous  research  studies,  was  originally  selected  for  its  high  imagery  content 
and  was  intended  to  stimulate  imagery  processing.  After  each  paragraph  students  were  referred  to  an 
answer  booklet.  The  answer  booklet  contained  the  instructions  for  each  treatment. 

EosttesL  A  68-item  postlest  measured  literal  comprehension.  A  typical  posttest  question  is: 
"A  hydrophone  network  set  up  bv  the  Navv  to  protect  (Chesapeake^  Bav  during  World  War  II  was 
temporarily  made  useless  hv  noise  made  hv  fish."  Since  the  original  72  item  posttest  was  unavailable, 
the  posttest  was  constructed  from  items  provided  by  Dr.  Ernst  Rothkopf3.  The  KR-21  reliability  for 
the  posttest  was  0.74. 

Summaries  and  analogies.  The  summaries  and  analogies  generated  d'iring  the  experiment  were 
analyzed  to  determine  the  degree  to  which  students  followed  instructions  during  the  experiment  These 
scores  were  analyzed  further  to  determine  the  extent  to  whicn  the  experimental  treatments  influenced 
posttest  achievement.  Each  analogy  or  summary  generated  according  to  directions  provided  during  the 
experiment  scored  two  points.  An  analogy  or  summary  which. only  partially  complied  with  directions 
scored  one  pcin^  Other  generations  did  not  receive  a  point  An  example  of  a  student  generated 
summary  is: 1  Sperm  whales  go  deep  down  in  >he  ocean  to  feed  on  animal  life.  Squids  inhabit  this 
region  also."  An  example  of  a  student  generated  analogy  is:  "Sir  John  Ross  has  shown  proof  that  thero 
is  some  form  of  life  'worms  at  a  depth  of  1000  fathoms.  Not  too  surprising-worms  like  a  dark  wet 
environment" 
Pesign 

"Hie  study  employed  a  2X2X2  factorial  design  with  two  appended  control  groups.  The  factors 
included  Grouping  (Indi  iduals,  Pairs),  Treatment  (Summary,  Analogy),  and  Ability  (High,  Low).  In 
one  control  group  students  'udicd  alone  and  read  each  paragraph  twice  before  continuing  to  read  the 
passage.  In  the  other  control  group  students  studied  in  pairs  and  recorded  comments  to  learn  as  much  as 
possible  from  each  paragraph,  but  were  not  directed  how  to  interact  or  as  to  the  nature  of  information  to 
record.  Dependent  measures  included  posOest  achievement  and  time  (  task.  Also,  students  in  the 
analogies  and  summaries  treatments  received  scores  for  writing  produced  during  the  experiment 
Procedures 

The  pre-test  was  completed  approximately  one  week  before  the  study.  Prior  to  the 
experiment,  students  were  randomly  assigned  to  work  individually  or  in  pairs,  and  to  one  of  two 
treatment  groups  or  two  control  groups  resulting  n  six  analysis  groups  (Individual  summary, 
Individual  analogy,  Individual  re-read,  Paired  summary,  Paired  analogy,  Un-scripted  pairs).  Students 
working  alone  received  identical  instructions  to  those  provided  in  a  previous  study  (Wittrock  & 
Alesandrini,  1990).  All  students  were  instructed  to  record  the  start  and  finish  time  of  the  experiment 
Students  assigned  to  Uw  Summary  treatment  were  instructed  to  read  and  review  a  paragraph  of  text  and 
then  to  summarize  each  paragr  iph  in  a  space  provided  hi  an  answer  booklet  A  summary  was  described 
as  consisting  of  one  or  two  sc  itenccs  that  state  the  main  idea  or  topic  of  a  paragraph.  Students  were 
asked  not  to  copy  sentence*,  directly  from  the  passage,  not  to  use  any  terminology  from  the  paragraph, 
and  10  write  the  scnterc^  in  their  own  words. 
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A  similar  statement  was  provided  for  students  in  the  analogy  treatment.  Students  were  asked 
to  review  each  paragraph  and  to  write  an  analogy  associating  the  content  from  the  text  with  things  that 
students  already  knew  about.  An  analogy  was  described  as  consisting  of  one  or  two  sentences  which 
clearly  related  the  new  ideas  to  familiar  things  or  experiences. 

Students  in  the  re-rcad  treatment  were  instructed  to  record  the  time  before,  and  again  upon 
completing,  reading  each  paragraph.  After  recording  the  finish  time,  students  were  instructed  to  re- 
read each  paragraph.  Students  were  also  informed  that  the  experiment  was  not  examining  speed 
reading  and  that  recording  the  time  helped  to  fulfill  an  important  experimental  condition. 

Students  assigned  to  the  paired  Summary  and  Analogy  treatments  received  identical 
instructions  to  the  corresponding  individual  treatments,  together  with  instructions  to  work  together 
to  form  analogies/  summaries  for  each  paragraph,  to  share  ideas  and  experiences,  and  that  each 
partner  should  write  an  analogy/summary  individually  in  their  answer  booklets.  Students 
working  in  the  paired  control  group  were  instructed  to  work  with  their  partner,  to  remember  as 
much  of  the  passage  as  possible,  and  to  enter  relevant  comments  into  an  answer  booklet  after 
reading  each  paragraph. 

Results 

Posttest  Achievement 

Posttest  means  and  standard  deviations  arc  contained  in  Table  1.  A  significant  difference  was 
found  for  Grc4ping,E(  1,67)=  7.73,  J2<.01.  Students  who  completed  the  instruction  individually  scored 
higher  than  students  who  completed  the  instruction  in  pairs  (means  21.73  and  17.63).  A  significant 
difference  was  also  found  for  Treatment,  E(l,67)=  4.75,  b<.03.  Students  who  completed  the  Summary 
treatment  scored  higher  than  students  who  completed  the  Analogy  treatment  (means  21.41  and  17.75). 
A  significant  interaction  was  found  between  Ability  and  Grouping,  E(1.67)=  5.10,  c<.03.  The 
interaction  is  shown  in  Figure  1.  Follow-up  comparisons  indicated  that  the  achievement  scores  for 
high  ability  studying  alone  were  higher  than  those  of  high  ability  students  studying  in  pairs  and 
students  working  alone.  No  statistically  reliable  differences  were  found  between  the  treatment  and 
control  groups. 


Insert  Table  1  and  Figure  1  About  Here 


No  differences  were  found  for  time  on  task.  However,  posttest  scores  were  divided  by  time  on 
task  to  produce  a  measure  of  learning  efficiency.  This  metric  produced  a  significant  difference  for 
Grouping  EO>67)-  12.22,  c<.001.  The  learning  rate  for  students  who  completed  the  instruction 
individually  was  higher  than  for  students  who  worked  in  pairs  (means  0.267  and  0.200).  A  significant 
difference  was  also  found  for  Treatment  E0.67)=  4.05,  £<.05.  Students  who  completed  the  Summary 
treatment  were  more  efficient  than  students  who  completed  the  Analogy  treatment  (means  0.253  and 
0.210). 

Summaries  and  Analogies 

Posttest  means  and  standard  deviations  are  contained  in  Tabic  2.  Significant  correlations  were 
found  between  posttest  achievement  and  summaries,  r£39)=  .3 18,  j2<.05,  and  posttest  achievement  and 
analogies,  i£36)=  .373,  £<.05.  Further  analyses  investigated  the  quantity  generated  individually  and  in 
pairs.  Separate  independent  t-tcsts  were  used  to  examine  the  quantity  of  analogies  and  summaries 
generated  in  paired  and  individual  treatments.  Analysis  of  the  summaries  produced  a  significant 
difference  for  Grouping,  1(37)=  2.402  p<0.05.  Students  working  alone  generated  significantly  more 
summaries  than  students  working  in  pairs  (means  87.1 1  and  79.50).  However,  the  analysis  of 
analogies  was  not  significant  *(34)=  0.72  p>0.05.  The  means  of  students  working  alone  (mean= 
40.11)  and  in  pairs  (mean=  42.78)  were  not  reliably  different 

Insert  Table  2  About  Here 


Discussion 

Students  working  alone  scored  significantly  higher  on  the  posttest  than  students  working  in 
pairs.  Moreover,  the  significant  interaction  between  Ability  and  Grouping  indicates  that  pairing 
impaired  high  ability  students  performance  when  learning  from  text,  but  did  not  affect  the  low  ability 
students.  Achievement  was  significantly  higher  for  high  ability  students  working  individually  (mean  = 
24.81)  than  in  groups  (mean  »  19.04).  However,  achievement  differences  between  low  ability  students 
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working  alone  (mean  «  17.88)  and  in  small  groups  (mean  »  17.70)  were  not  significant.  In  other 
words,  grouping  hampered  only  the  high  ability  students.  Furthermore,  pairing  reduced  instructional 
efficiency.  Individuals  completed  instruction  significantly  more  efficiently  than  collaborators. 

These  results  deviate  from  research  supporting  small  group  interventions.  Most  studies  report 
significant  advantages  gained  from  collaboration.  Furthermore,  although  some  studies  suggest  that  not 
all  students  benefit  from  collaboration  iWebb,  1982a;  Beane  &  Lemke,  1971),  several  studies  support 
the  efficacy  of  grouping  for  all  ability  groups.  For  example,  two  studies  reported  that  high,  average, 
and  low  ability  second  and  third  grade  students  demonstrated  higher  performance  after  collaborating  tl  an 
after  working  alone  (Yager,  Johnson,  &  Johnson,  1985;  Yager,  Johnson,  Johnson,  &  Snider,  1986). 
Similarly,  high,  average,  and  low  ability  students  demonstrated  higher  achievement  and  lower  math 
anxiety  after  completing  a  computer-based  integrated  learning  system  in  pairs  rather  alone  (Mevarcch, 
Silber,  &  Fine,  1991).  Hooper  (in  press),  also  reported  that  high  and  average  ability  fifth  and  sixth 
grade  students,  working  in  heterogeneous  or  homogeneous  ability  groups,  outperformed  their 
counterparts  working  alone  while  completing  a  computer-based  tutorial. 

Two  explanations  for  the  present  findings  are  suggested.  First,  students  working  in  groups 
may  have  experienced  the  free  rider  effect  (Kerr  &  Bruun,  1983).  The  free  rider  effect  is  a  phenomenon 
that  occurs  in  groups  when  individual  effort  is  presumed  to  be  unnecessary  due  to  the  perceived 
competence  of  other  members.  The  resulting  reduced  effort  is  manifested  in  reduced  group  performance. 
In  the  present  study,  paired  students  may  have  invested  less  effort  in  the  leamir g  task  than  students 
working  alone,  relying  instead  on  their  partners  to  furnish  the  required  informa Jon.  This  conjecture  is 
supported,  in  part,  by  the  count  of  the  summaries  generated  during  the  experiment.  Means  for 
summaries  were  significantly  higher  for  students  working  alone  than  in  pairs.  Further  inspection 
reveals  significant  differences  between  high  ability  students1  generations,  but  only  small  differences 
between  the  low  ability  students1  generations.  High  ability  students  may  have  believed  their  partners 
were  competent  at  forming  effective  summaries  and  consequently  invested  less  personal  effort  in  the 
task. 

Second,  students  may  have  interacted  ineffectively.  Researchers  generally  agree  that  the 
efficacy  of  small  group  learning  is  linked  to  the  nature  of  intra-group  interaction  (Webb,  1982b; 
Hooper,  in  press).  Effective  interaction  stimulates  cognitive  effort  which  in  turn  mediates  deeper 
processing  (Salomon,  1984;  Wittrock,  1990).  However,  appropriate  small  group  interaction  often 
does  not  occur  naturally,  and  simply  encouraging  college  students  to  work  together  may  be 
ineffective  (Carrier  &  Sales,  1987).  Unlike  school  children  who  socialize  frequently,  many  college 
students  have  little  social  contact  with  their  class  peers  and  may  not  interact  effectively  when  asked 
to  do  so. 

Another  finding  from  the  present  study  is  that  working  with  a  partner  did  not  help  students  to 
generate  effective  analogies  and  did  not  enhance  posttest  performance.  Students  in  the  Summary 
treatment  demonstrated  higher  posttest  performance  and  greater  instructional  efficiency  than  students  in 
the  Analogy  treatment  Furthermore,  means  for  the  count  of  the  summaries  generated  during 
instruction  were  more  than  double  the  means  for  the  analogies  count.  Although  direct  comparison 
must  be  made  with  caution,  these  differences  indicate  that  students  in  the  Analogy  treatment 
experienced  difficulties  completing  their  task.  Means  for  achievement,  generations,  and  time-on- task 
for  each  of  the  text  processing  strategies  included  in  this  and  the  Wittrock  and  Alesandrini  (1990)  study 
are  included  in  Table  3. 


The  present  study  failed  to  replicate  Wittrock  and  Alesandrini's  (1990)  finding  that  students 
using  generative  learning  strategics  demonstrated  higher  posttest  performance  than  students  who  rc-rcad 
a  passage.  Little  evidence  exists  to  explain  this  finding,  although  some  differences  existed  in  the 
administration  of  the  original  and  the  replication  studies.  Although  both  studies  employed  identical 
stimulus  materials  the  posttest  used  in  the  present  study  differed  from  the  original  posttest.  Also, 
students  in  the  original  study  completed  the  experimental  treatments  in  sessions  oi  4-6  people.  In  the 
present  study,  the  treaUnents  were  administered  in  sessions  of  approximately  30  people. 

Effective  interaction  may  require  interdependence  among  group  members  and  guidance  in  using 
interaction  techniques.  Salomon  (1989)  suggested  that  the  cognitive  benefits  of  group  learning  are 
limited  to  the  extent  that  students  are  interdependent  and  that  group  members  must  commit  to  common 
goals  and  agree  to  coordinate  and  invest  effort  in  order  to  experience  die  cognitive  benefits  of  group 
learning.  Salomon  also  proposed  that  interdependence  is  likely  to  be  low  during  short  research  studies 
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involving  subjects  with  little  previous  interpersonal  knowledge  or  interaction  and  that  under  such 
conditions  small  groups  may  be  ineffective  learning  environments. 

Future  research  should  focus  on  teaching  students  to  interact  effectively  in  groups  and  to 
generate  analogies.  Without  prior  practice,  many  students  are  unlikely  to  generate  effective 
analogies.  Furthermore,  research  examining  methods  to  stimulate  intra-group  interaction  suggests 
that  students  must  be  guided  to  interact  effectively.  Without  guidance  students  working  in  small 
groups  often  model  teaching  techniques  that  are  familiar,  but  unrelated  to  group  success  (McKcllar, 
1986). 

Notes 

1.  Weshsler  Adult  Intelligence  Scale.  Copyright  ©  1955  by  The  Psychological  Corporation. 
Reproduced  by  permission.  All  rights  reserved 

2.  From  The  Sea  Around  Us.  Revised  Edition,  By  Rachel  L,  Carlson.  Copyright  ©  1950, 1951, 
1961  by  Rachel  L.  Carlson;  renewed  1979  by  Roger  Christie.  Reprinted  by  permission  of 
Oxford  University  Press,  Inc. 

3.  The  authors  wish  to  thank  Dr.  Ernst  Rothkopf  of  Teachers  College,  Columbia 
University,  for  supplying  the  test  used  in  this  study. 
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Figure  1.  Interaction  between  Ability  and  Grouping 
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Tabic  1:  A^'cvement  and  Timc-on-task  Means  and  S  landed.  DgyMoM 


Group  Posttest  Time-on-task 

Ability 


Individuals 

High 

Re-read 

M 

23.20 

74,80 

(n=10) 

SD 

9.69 

1 1 .92 

Summary 

M 

27,00 

82.40 

(n=10) 

SD 

6.77 

12.46 

Analogy 

M 

23.29 

O  C  A'l 

86.43 

(n  =  7) 

SD 

0.02 

1  A  KA 
14.  j4 

Total 

M 

Of   A  H 

25.4  / 

Si  A  (ML 

o4.0o 

(n=17) 

SD 

1  J  i  u  / 

Lew 

Re-rcad 

M 

19.00 

91.25 

(n=10) 

SD 

8,70 

17.02 

Summary 

M 

19.56 

88.78 

(n=9) 

SD 

5 ,39 

1  A  HA 

14.t>4 

Analogy 

M 

17.73 

80.64 

(n  -  11) 

SD 

1  A  £Q 

1  4,08 

TotaJ 

\A 

M 

1  c  JJ 

SLA  in 

(n=20) 

SD 

Pairs 

Comments 

M 

21.38 

(n«10) 

SD 

8.1! 

13.95 

Summary 

M 

IV. ID 

(n=t3) 

SD 

6.49 

9.24 

Analogy 

M 

1  C  C£ 

90. 2z 

(n  =  9) 

SD 

6  19 

» 8.75 

iVi 

1  7  Aft 

(n=22) 

SD 

6.48 

13.92 

lm 

Comments 

M 

18.50 

85.10 

(n=10) 

SD 

9.11 

8.39 

Summary 

M 

20.00 

90.14 

(n=7) 

SD 

6.88 

13.23 

Analogy 

M 

15.67 

95.22 

(n-9) 

SD 

6.60 

21.78 

Tpt^l 

M 

17.56 

93.00 

(n=16) 

SD 

6.86 

18.16 
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Table  1  (cont):  Main  Effects  Summary 


Posttest 


Time 


group 

Individuals  (n=37) 

Pairs  (n=38) 


Ability 

High  (n=39) 

Low  (n=36) 


Summary  (n=39) 
Analogy  (n-~36) 


M 

SD 

M 

SD 


M 

SD 

M 

SD 


M 

SD 
M 

SD 


21.73 
7.37 

17.63 
6.55 


21.08 
7.52 

18.11 
6.65 


21.41 
6.99 

17.75 
7.07 


84.19 
13.87 
89.13 
15.97 


85.3.3 
13.43 
88.17 
16.75 


85.67 
12.11 
87.81 
17. S3 


Table  2 

Summary  and  Anabgy  Ratings  Means  and  Standard  Deviation 


Grouping 


Generaiion 


Ability 


Summary 


Pair 


Low 


iidiyidua) 


High  M 


Low 


Analogy 


M 

77.6 

43.6 

SD 

12.8 

10.24 

n 

(14) 

(9) 

M 

83.8 

42.0 

SD 

9.4 

10.3 

v 

(6) 

(5) 

M 

79.5 

42.8 

SD 

12.1 

9.9 

n 

(20) 

(18) 

M 

87.3 

46.3 

SD 

7.6 

7.2 

n 

(U» 

(7) 

M 

8«.9 

36,2 

r»D 

6.4 

i3.4 

n 

(9) 

(11) 

M 

87.1 

40.1 

SO 

'.2.2 

rt 

(1.9) 

(IS) 
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Tabic  3 

Means  and  Standard  Deviations  for  Original  and  Replication  Studies 


Treatments 


Individuals 

Pairs 

Learning  &  abilities 

Read 

Generate 

Generate 

Generate 

Generate 

Generate 

text 

summaries 

analogies 

comments 

summaries 

analogies 

Learning  (72  items) 

(n=19) 

(n=19) 

(n-19) 

(O)Mcan 

22.4 

29.8 

27.2 

(O)SD 

(9  2) 

(6  3) 

Ixaming  (68  itc  ms) 

(n=18) 

(n=19) 

(n«18) 

(n=18) 

(rrfO) 

(n=18) 

(R)Mcan 

21.33 

23.47 

19.89 

19.78 

19.45 

15.61 

(R)SD 

(9.25) 

(7.01) 

(7.40) 

(8.55) 

(6.46) 

(6.20) 

Generations  ( 100  maximum) 

(O)Mean 

* 

98.7 

75.4 

(0)SD 

(2.4) 

(23.3) 

Generations  (100  maximum) 

(R)Mean 

87.12 

40.lt 

79.50 

42.78 

(R)SD 

6.88 

12.25 

* 

12.06 

9.99 

Time  to  Ixarn  (minutes) 

(O)Mean 

76.1 

86.3 

85.2 

(0)SD 

(21.6) 

(18.6) 

(15.3) 

Time  to  Learn  (minutes) 

(R)Mean 

82.11 

85.41 

82.89 

86.78 

85.90 

92.72 

(R)SD 

(16.28) 

(13.55) 

(14.48) 

(11.01) 

(10.93) 

(19.88) 

Noic:  O  represents  scores  from  the  original  study.  R  represents  scores  from  the  replication  study. 
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Introduction 

Problem  identification  is  arguably  the  most  crucial  aspect  of  the  needs  assessment 
process  in  instructional  development.  Yet  little  attention  has  been  given  by 
instructional  developers  to  clearly  delineating  what  is  meant  by  the  term  'problem'. 
This  is  troublesome  given  that  many  definitions  of  the  word  problem  exist.  For 
example,  the  Oxford  English  Dictionary  describes  no  fewer  than  six  different  kinds 
of  problems.  This  suggests  that  the  way  one  identifies  a  problem  shapes  the  way 
one  views  the  problem  solving  process,  and  in  turn  influences  the  kinds  of 
problems  found  during  a  needs  assessment.  We  contend  that  getting  clearer  about 
how  one  identifies  problems  is  crucial  for  a  field  that  has  been  described  as  being 
primarily  a  problem  solving  process  (AECT  1977,  Romiszowski,1981). 

A  potentially  useful  approach  to  problem  clarification  may  be  the  use  of  evaluation 
models.  There  are  many  similarities  between  program  evaluation  and  needs 
assessment  in  instructional  development.  Many  of  the  functions  that  occur  in 
program  evaluation  have,  historically  at  least,  occured  in  needs  assessment.  Clients 
are  consulted,  data  is  gathered,  reports  are  written,  and  recommendations  made. 
But  there  is  one  important  and  potentially  useful  difference  -  evaluators  are  often 
explicit  about  the  value  perspective  they  bring  to  a  project  through  the  evaluation 
model(s)  they  consciously  adopt.  A  closer  examination  of  these  models  may  yield 
useful  frameworks  for  instructional  developers  as  they  tackle  a  needs  assessment. 

The  purpose  of  this  paper,  then,  is  to  develop  different  approaches  to  clarifying 
problems  in  the  instructional  development  process.  This  will  be  done  by:  1) 
defining  what  a  problem  is  and  what  types  of  problems  typically  face  instructional 
developers;  2)  presenting  a  rationale  for  suggesting  the  use  of  evaluation  models  for 
finding  development  problems;  3)  suggesting  how  the  use  of  a  particular  evaluation 
approaches  may  result  in  the  finding  of  different  problems. 

What  Is  A  Problem? 

The  concept  of  problem  is  very  complex.  Although  we  often  take  its  meaning  for 
granted.  Academicians  tend  to  describe  the  term  in  ways  that  may  be  foreign  to  us. 
One  example  of  this  is  Clark's  (1979)  definition: 

a  situation  resulting  from  the  interaction  or  juxtaposition  of  factors 
which  yields  a  perplexing  or  enigmatic  state,  an  undesirable 
consequence,  or  conflict.  The  juxtaposition  can  take  the  form  of 
provocative  exceptions,  contradictory  evidence,  a  knowledge  void, 
or  a  theoretical  action-knowledge  conflict  (as  cited  in 
Jorgenson,198 1  p. 9), 

The  difficulty  with  a  definition  like  Clark's  is  that  it  is  not  easily  understood  by 
practising  instructional  developers  or  their  clients  and  is  open  to  various 
interpretations,  Clearly  understanding  some  of  these  interpretations  would  be 
helpful  for  individuals  engaged  in  the  instructional  development  process.  A 
dictionary  definition  is  a  useful  starting  point  in  identifying  the  different 
interpretations  of  the  term  problem. 
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The  Oxford  English  Dictionary  (1989),  for  example,  identifies  six  kinds  of 
problems: 

problem  -  a  thing  thrown  or  put  forward;  hence,  a  question  propounded 
for  a  solution,  a  set  task,  a  problem. 

1.  A  difficult  or  puzzling  question  proposed  for  a  solution;  a  riddle;  an 
enigmatic  statement. 

2.  a,  A  question  proposed  for  academic  discussion  or  scholastic 

disputation. 

b.  Logic.  The  question  (expressed,  or,  more  usually,  only  implied) 
involved  in  a  syllogism,  and  of  which  the  conclusion  is  the  only 
solution  or  answer. 

3.  a.   A  doubtful  or  difficult  question:  a  matter  of  inquiry,  discussion, 

or  thought;  a  question  that  exercises  the  mind. 

b.  Problematic  qualify;  difficulty  of  solution. 

c.  As  the  second  element  in  various  Combs,  and  collocations 
describing:  (a)  a  supposedly  insoluble  quandary  affecting  a 
specified  group  of  people  or  a  nation;  (b)  a  real  or  imagined 
chronic  personal  difficulty,  ax  credibility,  drink,  health,  weight 
problem. 

d.  In  various  colloq.  phrases,  as  no  problem,  simple  easy,  'the 
question  does  not  arise';  that's  your  (his,  etc.)  problem,  used  to 
disclaim  responsibility  or  connection, 

4.  Geom.  A  proposition  in  which  something  is  required  to  be  done: 
opposed  to  theorem.. 

5.  Physics  and  Math..  A  question  or  inquiry  which  starting  from 
seme  given  conditions  investigates  some  fact,  result,  or  law. 

6.  Chess  .  An  arrangement  of  pieces  upon  the  chessboard  for  play  in 
accordance  with  the  rules  of  the  game  or  other  prescribed 
conditions,  in  which  the  player  is  challenged  to  discover  the  method 
of  accomplishing  a  specified  result. 

The  number  and  substance  of  these  definitions  suggests  that  there  are  different 
kinds  of  problems  and  different  uses  of  the  word  'problem'.  If  one  accepts  that 
there  are  different  kinds  of  problems,  and  different  uses  of  the  word  problem, 
significant  questions  arise/what  problems  are  most  likely  to  find  their  way  into  the 
practice  of  the  professional  who  solves  problems?'  Are  professionals  who  solve 
problems  aware  that  different  kinds  of  problems  exist?  Are  they  trained  to  solve 
different  kinds  of  problems?  Are  certain  kinds  of  professions  only  concerned  with 
particular  kinds  of  problems? 

It  would  seem  that  all  professionals  must  address,  at  one  time  or  another,  problems 
of 'academic  discussion  and  scholastic  disputation'  and  'difficult  questions  that 
exercise  the  mind'.  Some  professionals,  such  as  medical  doctors  or  architects,  look 
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to  attain  some  specified  result,  although,  it  would  seem  that  they  havo  a  number  of 
different  avenues  or  methods  available  to  do  this  (definition  number  6).  Others, 
such  as  chemists  or  physicist,  seek  to  investigate  a  mathematical  or  scientific  law  or 
result  (definition  number  5). 

Why  should  instructional  developers  be  concerned  with  the  concept  of  problem'/ 
Simply  put,  because  problem  solving  is  an  integral  part  of  what  instructional 
developers  do  (Romiszowski,  1981;  Dick  &  Carey,  1990).  This  emphasis  can  be 
clearly  seen  in  the  most  recent  definition  put  forth  by  the  Association  for 
Educational  Communications  and  Technology,  the  largest  professional  orr^ization 
for  instructional  developers  in  the  United  States: 

Educational  technology  is  a  complex,  integrated  process,  invoMng 
people,  procedures,  ideas,  devices  and  organization,  for  analyzing 
problems  and  devising,  implementing,  evaluating  and  managing 
solutions  to  those  problems,  involved  in  all  aspects  of  human 
learning  [emphasis  added]  \  AECT.  '77,  o.  1). 

How  Instructional  Develops  See  Problems 

But  what  kind  of  problems  do  professional  instructional  developers  typically  deal 
with?  It  is  our  contention  that  tl;  ■  field  of  instructional  development  treats  problems 
as  essentially  solvable.  Problem  -  are.  in  tact,  generated  to  be  solved  in 
instructional  developmen;  :-od  needs  assessment  is  the  stage  in  the  instructional 
development  process  wh;*re  a  problem  is  identified.  Kaufman  and  English  outline 
important  considerations  in  a  needs  assessment: 

the  formal  harvesting  collet '.on,  and  listing  of  needs  in  priority 
order,  and  selecting  the  need',  of  highest  priority  for  action.  This 
process  include:.  «.Sie  partners  in  planning,  which  in  education  are 
learners,  the  implements,  and  the  society.  It  requires  that  there  be 
a  consensus  of  the  partners  in  the  prioritization  of  needs,  and  it 
stroPfjly  'K  t/  s  th'U  the  process  include  any  additional  external 
refereVr  o''surv;ai  and  contribution  when  determining  need 
priorities.  (Kaufman  and  English,  1979,pp.  343-344). 

But  vvfi.nt  exactly  is  a  need'.'  The  most  common  definition  of  'need'  in  the  literature 
of  N.'dueatio-.ai  technology  and  instructional  development  is  that  it  is  a  discrepancy  or 
between  the  current  situation  and  a  desired  one-"a  discrepancy  between  what  is 
And  what  is  required"  (Kaufman.  1968,  p.  415).  Problems  in  instructic  lal 
development  are  needs.  Needs  are  gaps  in  performance  outcomes,  and  these 
ot;  somes  are  to  be  stated  in  measurable  terms  (Kaufman  &  English.  1979). 
i\aufman  and  English  relate  problems  to  needs  as: 

Problems  are  needs  selected  for  closure  -  if  there  is  no  gap.  there  is 
no  problem.  The  problem  identification,  properly  done,  starts  with 
an  appropriate  assessment  of  needs.  A  Needs  Assessment  is  the 
determination  of  documentable  and  important  gaps  between  current 
outcomes  and  desired  outcomes,  and  the  placing  of  those  gaps  in 
priority  order  for  closure  (Kaufman  &  English,  1979.  p. 24) 

Following  this  line  of  thinking,  problems  in  instructional  development  are  not 
questions  for  scholarly  inquiry  or  academic  disputation.  The  identification  of  a 
problem  in  instructional  development  is  a  call  for  action  (closure):  it  requires  that 
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something  be  done.  Further,  it  is  a  call  for  a  technical  sort  of  action  --  action 
following  a  set  of  pre  specified  guidelines.  Viewed  in  this  way,  instructional 
development  problems  are  perhaps  most  akin  to  the  Oxford  English  Dictionary 
problem  type  four  "a  proposition  in  which  something  is  required  to  be  done"  and 
type  six  "an  arrangement  of  pieces  upon  the  chessboard  for  play  in  accordance  with 
the  rules  of  the  game  or  other  prescribed  conditions,  in  which  the  player  is 
challenged  to  discover  the  method  of  accomplishing  a  specified  result". 

The  latter  definition  of  problem,  one  couched  in  the  analogy  of  a  chess  game,  is 
useful  in  helping  understand  how  instructional  developers  view  the  problem  solving 
process.  The  analogy  suggests  four  things  about  the  way  problems  are  solved  by 
instructional  developers.  1)  There  are  a  set  of  rules  under  which  the  activity  must 
operate.  2)  There  is  ultimately  one  desired  outcome:  the  problem  that  has  been 
identified  must  be  solved.  3)  While  there  are  rules,  there  are  nevertheless,  many 
ways  to  solve  an  instructional  development  problem.  4)  It  is  clear  when  the 
instructional  development  problem  has  been  solved. 

The  Problems  With  Traditional  ID  Problems 

There  are  two  important  ways  in  which  the  chess  analogy  to  instructional 
development  breaks  down.  One  is  that,  in  chess  the  outcome  is  measurable.  It  is 
easily  ascertained,  according  to  the  rules,  whether  one  has  won,  lost,  or  drawn.  In 
instructional  development,  outcomes  are  construed  so  that  they  can  be  measured. 
This  requires  interpreting  the  situation  so  that  the  desired  state  can  be  measured. 

The  other  is  that  the  problem  of  chess  exists  before  two  players  begin  the  game.  !t 
is  a  given.  It  is  not  open  to  the  interpretation  of  the  players.  The  problem  exists  a 
priori.  In  contrast,  however,  problems  in  instructional  development  are  not  a 
given.  They  are  consciously  generated  during  the  instructional  development 
process.  An  instructional  development  problem  is  socially  construed.  It  is 
generated  by  the  individuals  engaged  in  the  instructional  development  process.  An 
instructional  development  problem,  therefore,  can  not  exist  a  priori. 

Concerns  have  been  raised  about  the  activities  and  intentions  of  needs  assessment. 
First,  can  needs  assessment  be  responsive  to  the  intent  of  the  planning  partners  or 
clients  if  it  is  converting  the  data  and  information  that  was  gathered  to  determine 
educational  goals  into  precise  and  measurable  objectives?  It  seems  that  a  certain 
degree  of 'distortion'  is  inevitable  when  operationalizing  goals  into  measurable 
objectives  (Jorgenson,1981).  Can  the  intent  of  the  planning  partners  or  clients  be 
truly  represented  and  maintained  after  this  process  has  taken  place? 

Even  Robert  Mager,  the  person  who  may  be  most  responsible  for  making  the  use  of 
behavioral  objective  popular,  suggests  that  goals  cannot  easiK  be  translated  into 
measurable  objectives: 

Though  it  is  not  always  possible  to  describe  an  intent  clearly,  and 
though  it  is  not  always  possible  to  determine  whether  an  important 
outcome  has  been  achieved,  instruction  can  only  be  considered 
elegant  to  the  extent  that  it  effectively  and  efficiently  facilitates 
achievement  of  instructional  intent. (p. 20  as  cited  in 
Jorgenson,  198 1 ) 

If  the  original  intent  of  the  planning  partners  or  clients  is  distorted,  how  can 
instructional  development  meet  their  real  needs'.'  When  faced  with  this  question 
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instructional  developers  often  appeal  to  a  consensus  of  the  planning  partners. 
Gaining  consensus  among  the  individuals  engaged  in  the  instructional  development 
process  may  be  difficult.  Can  the  instructional  developer  gain  a  consensuses  about 
output  of  the  information  interpretation  process?  Even  if  such  a  consensus  were 
possible  to  attain  (gaining  a  similar  interpretation),  can  the  instructional  developer 
expect  to  gain  a  consensus  on  the  prioritization  of  these  interpretations  (a  question 
of  attaining  agreement  on  values)?  Cuba  and  Lincoln  argue  that: 

...recent  writings  in  responsive,  naturalistic  modes  of  inquiry  raise 
the  issue  that  multiple  realities  and  pluralistic  value  perspectives 
render  consensus  an  unreal  expectation.  If  it  is  achieved,  it  may  be 
an  inauihentic  construction  superimposed  upon  the  complex  rtalities 
of  the  situation.  Certainly  generalizations  are  possible,  and  essential 
to  decision-making  and  action,  but  it  is  important  to  acknowledge 
that  generalizations  about  human  needs,  wants  and  goals  have 
inextricably  subjective,  time-bound  dimensions  to  them.  "Needs 
have  no  objective  reality."  (Jorgenson.1981  p.  206  citing  Guba  and 
Lincoln,  1981,  p.  1) 

This  idea  applies  to  both  the  planning  partners  and  the  instructional  developer. 
"Needs  are  not  objectively  'real'  and  open  to  discovery  by  scientific. .value-free 
methodologies."  (Guba  and  Lincoln,  1981,  p.  1).  Needs  are  grounded  in  the  value 
systems  of  the  planning  partners  or  clients  and  the  instructional  developer.  It  a 
need  is  the  'gap'  between  where  we  are  and  where  we  want  to  be,  then  a  need  is 
little  more  than  a  way  of  saying  we  want  something.  What  we  want  is, 
presumably,  something  we  value. 

Guba  and  Lincoln  insist  that: 

It  is  time  for  reassessment  of  needs  assessment,  and  for  the 
acceptance  of  a  methodology  which  deals  with  the  values  issue 
squarely,  takes  values  overtly  into  account,  and  understands  needs 
for  what  thev  are:  value  expressions.  (Guba  and  Lincoln,  1981, 
p.20) 

We  agree  with  Guba  and  Lincoln  and  recommend  that  the  role  of  values  in  the 
problem  clarification  process  be  acknowledged  and  studied.  If  the  role  of  values  is 
acknowledged  then  the  developer  and  the  planning  partners  or  clients  will  be  more 
likely  to  attend  to  their  values  as  the  process  unfolds. 

Using  Evaluation  Models  To  Clarify  Netd 

How  can  we  address  Guba  and  Lincoln's  challenge  to  deal  with  the  problem  of 
values  in  the  needs  assessment  process.  Evaluation  models  may  offer  the 
instructional  developer  some  useful  insights.  First,  many  of  the  functions  that 
occur  in  program  evaluation  also  occur  in  needs  assessment.  Second,  and  most 
importantly  evaluators  seek  frequently  to  improve  what  is    a  notion  that  for 
evaluators  consciously  implies  a  value  judgement. 

Many  different  evaluation  models  are  described  in  the  literature;  nevertheless,  a 
measure  of  consensus  does  exist  over  a  number  of  major  evaluation  approaches  -- 
some  characterized  by  a  single  evaluation  model  and  some  by  several  models. 
Worthen  and  Sanders  (1987")  provide  a  good  summary  of  several  of  the  major 
evaluation  approaches.  They  identify  six  basic  approaches:  objectives-oriented. 
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management  oriented,  consumer  oriented,  expertise  oriented,  adversary  oriented, 
and  naturalistic  and  participant  oriented.  What  follows  is  a  cursory  description  ot 
each  evaluation  approach  and  their  possible  use  in  needs  assessment. 

Qbjective-oriented  Approach 

The  objective-oriented  approach  plays  a  prominent  role  in  the  development  of 
educational  evaluation.  In  fact,  the  field  of  educational  evaluation  was  initially 
conceptualized  in  terms  of  behavioral  objectives  and  determining  a  learner's  ability 
to  meet  these  objectives  (Worthen  and  Sanders,  1987).  Differences  between  stated 
behavioral  objectives  and  observed  behavior  are  seen  as  symptomatic  ot 
deficiencies  in  the  educational  innovation  being  evaluated.  Deficiencies  would 
ultimately  lead  to  changes  in  the  innovation  in  an  effort  to  ensure  that  eventually  the 
terminal  behavior  matched  the  stated  objectives  of  the  innovation. 

The  objective  oriented  approach  probably  has  the  greatest  affinity  with  the  problem 
identification  approach  presently  used  in  instructional  development  and  hence  is  not 
useful  in  suggesting  new  ways  of  problem  clarification.  As  previously  mentioned, 
the  prevailing  model  adhered  to  by  instructional  developers  during  problem 
clarification  seeks  to  characterize  differences  between  present  behavior  and  desired 
behavior  in  terms  of  behavioral  objectives  which  form  the  basis  for  the  subsequent 
design  of  instructional  materials. 

In  many  ways  it  can  be  argued  that  th-  other  evaluation  approaches  discussed  below 
have  been  developed  in  an  attempt  to  address  the  perceived  liabilities  of  the 
objective  oriented  approach.  For  this  reason  the  subsequent  approaches  would 
appear  to  hold  more  promise  for  suggesting  new  ways  of  clarifying  problems. 

Management-oriented  Approach 

Management  oriented  approaches,  like  the  CIPP  (Context,  Input,  Process,  and 
Product)  model  for  example,  are  among  the  most  widely  used  evaluation 
approaches.  The     mary  concern  of  these  kinds  of  evaluations  is  to  provide 
information  that  is  of  use  to  decision  makers.  Thus  the  evaluator  begins  by  clearly 
identifying  the  decisions  a  manager  must  make  and  then  gathers  information  that 
will  help  the  manager  make  these  decisions. 

Different  kinds  of  evaluations  have  been  suggested  for  different  kinds  of  decisions. 
The  CIPP  model,  for  example,  describes  context  eva'uations  (an  evaluation  that 
helps  clarify  the  setting,  goals  and  objectives  of  an  intervention),  input  evaluations 
(an  evaluation  that  clarifies  procedural  and  budgetary  plans  of  an  intervention), 
process  evaluation  (an  evaluation  that  identifies  and  corrects  problems  of  an 
intervention  while  it  is  in  progress),  and  product  evaluation  (an  evaluation  that 
clarifies  whether  to  continue  or  modify  an  existing  intervention  and  to  present  a 
clear  record  of  how  the  intervention  works). 

Romiszowski  (1981)  has  suggested  that  the  instructional  development  process  is 
very  much  like  the  one  laid  out  in  the  CIPP  model  -  and  that  a  needs  ussessmet.'  is 
very  much  like  the  "context"  evaluation  presented  in  the  CIPP  model.  Certainly, 
the  description  of  a  context  evaluation  offered  by  Stuftlebeam  (1968),  sounds  very 
much  like  what  an  instructional  developer  does  during  a  needs  assessment: 

Context  evaluation  would  be  used  when  a  project  is  first  being  planned. 
The  major  objective  of  context  evaluation  is  to  define  the  environment  where 
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change  is  to  occur,  the  environment's  unmet  needs  and  opportunities  for 
change.  Information  from  ccntext  evaluation  leads  ultimately  to  the 
establishment  of  program  goals  and  objectives. 

How  might  a  context  evaluation  be  undertaken  to  ensure  that  the  "value  systems  of 
the  planning  partners  and  the  instructional  developer"  are  identified?  It  could  be 
argued  that  a  thorough  context  evaluation  is  much  like  an  anthropological  study  - 
one  that  does  not  merely  describe  a  context  but  a  culture  which  is,  by  definition, 
value  laden.  This  idea  is  not  new.  An  anthropological  approach  to  ihe  study  of 
values  in  the  instructional  development  process  has  been  put  forward  by  Yeaman, 
(1990).  The  methods  of  the  anthropologist  may  help  to  identify  ,  describe,  and 
attend  to  the  values  of  the  instructional  developer,  client,  and  stakeholder.  Such  an 
approach  may  also  give  the  instructional  developer  insights  into  how  the  needs 
assessment  process  itself  influences  these  different  value  systems. 

Naturalistic  and  Participant  Oriented  Approach 

Naturalistic  and  participant  oriented  evaluation  approaches  squarely  place  the 
emphasis  of  evaluation  on  describing  and  understanding,  as  fully  as  possible,  how 
an  intervention  works  within  a  specific  context  Not  surprisingly,  therefore,  the 
methods  and  tools  of  the  anthropologist  and  very  useful.  This  approach  is  in  stark 
contrast  to  an  objectives  oriented  approach.  Understanding  is  achieved  through  a 
pluralistic  inductive  approach  to  evaluation.  That  is.  the  concerns  of  key 
stakeholders  rather  than  decision  makers  are  given  particular  attention.  Also  the  use 
of  more  naturalistic  and  participant  oriented  data  collection  n  ethods  is  emphasized. 
First  hand  experience  is  highly  valued  and  the  existence  of  multiple  realities  is 
readily  accepted. 

The  naturalistic  and  participant  oriented  evaluation  approaches,  in  our  opinion, 
maybe  especially  well  suited  to  problem  clarification  in  instructional  development. 
A  naturalistic/participant  oriented  instructional  analysis  would  be  concerned 
primarily  with  describing  and  understanding  a  particular  learning  environment.  It 
would  place  a  heavy  emphasis  upon  clearly  understanding  and  being  sensitive  to  the 
concerns  of  learners  themselves  rather  than  the  needs  of  administrators  or 
managers.  Many  of  the  data  gathering  methods  used  in  such  an  approach  show  up 
in  subsequent  evaluation  approaches.  For  example,  the  use  of  in  depth  interviews, 
focus  groups,  and  panel  discussions,  are  endemic  to  the  expertise  oriented  and 
adversarial  oriented  evaluation  approaches  as  well.  An  naturalistic  and  participant 
oriented  evaluation  approach  to  needs  assessment  would  also  rely  heavily  upon 
detailed  observations  of  a  settings.  These  "field  notes"  would  help  the  developer 
assess  the  values  of  the  key  players  involved  in  the  process 

Expertise-oriented  Approach 

Arguably  the  oldest  form  of  evaluation,  an  expertise-oriented  approach  relies  on  the 
knowledge  and  wisdom  of  qualified  experts  to  assess  the  worth  of  something. 
Review  panels,  professional  accreditation  committees,  movie  and  theater  critics  are 
all  examples  of  evaluations  made  by  an  expert  or  a  group  of  experts.  Expert 
reviews  vary  widely  with  respect  to  the  process  followed.  Some  expert  evaluations 
make  explicit  the  criteria  used  to  judge,  time  lines  used  in  making  judgments, 
conflicting  opin    *s  among  experts,  and  the  credentials  of  the  experts,  while  others 
make  none  of  these  points  explicit.  For  example,  a  panel  of  supreme  court  judges 
would  be  an  example  of  a  case  where  all  of  these  points  are  made  explicit  when  the 


33;:  35S 


8 


judgement  is  handed  down.  A  dissertation  committee  would  be  a  case  where  little 
additional  information  is  given  as  to  how  a  passing  or  failing  grade  was  determined. 

An  "expert"  needs  assessment  could  be  set  up  in  much  the  same  way  as  an  expert 
evaluation.  For  example,  a  panel  of  experts  could  be  asked  to  investigate  a  setting 
and  suggest  reasons  students  or  employees  are  not  performing  adequately.  Expert 
panels  made  up  of  individuals  familiar  with  the  setting  being  studied,  could  provide 
valuable  information  about  the  context  --  internal  conflicts,  budget  constraints  etc. 
Blue  Ribbon  panels  also  might  give  an  instructional  development  initiative  added 
weight  and  credibility  to  those  involved  who  may  be  resistant.  An  expert  panel 
would  also  readily  allow  for  a  range  of  input  and  opinion  while  giving,  possibly, 
key  players  a  greater  sense  of  ownership  in  the  process.  A  problem  clarification 
process  patterned  after  a  supreme  court  deliberation  may  be  useful  as  well.  One 
would  be  left  with  several  expert  views  of  what  the  instructional  problem  or 
problems  might  be  as  well  as  expert  assessments  of  the  value  systems  of  the  client 
and  and  how  these  shape  a  determination  of  the  instructional  problems.  While  an 
overall  decision  may  be  reached,  the  panel  is  not  constrained  to  reach  consensus. 
This  may  encourage  multiple  points  of  view  and  perspectives. 

Consumer-oriented  Approach 

The  consumer  oriented  evaluation  approaches  were  developed  so  that  individuals  or 
agencies  could  readily  assess  the  relative  merit  of  a  product  they  were  planning  to 
buy.  Rating  checklists  are  a  typical  method  of  undertaking  these  sons  of 
evaluations.  A  set  of  criteria  are  created  for  a  particular  type  of  product  (criteria  for 
a  car  might  include:  horsepower,  breaking  distance,  and  cost) .  Several  examples 
from  a  class  of  products  are  tf..,n  compared  with  respect  to  these  criteria.  A 
consumer  either  uses  the  checklist  to  assesses  a  product's  relative  strengths  and 
weaknesses  or  consults  a  report  where  someone  else  has  actually  rated  a  product 
using  the  evaluation  criteria.  The  emphasis  in  this  type  of  evaluation  is  to 
communicate  in  a  simple  straight  forward  manner  whether  a  particular  product  is 
worth  purchasing  or  not. 

A  consumer  oriented  approach  to  problem  clarification  suggests  a  check  list  of 
concerns  that  would  be  addressed  during  the  instructional  development  process. 
This  sort  of  checklist  could  serve  as  a  starting  point  in  a  needs  assessment.  A 
properly  created  checklist  might  ensure  that  developers  consider  a  number  of 
different  kinds  of  problems.  The  most  important  question  in  generating  a  checklist"- 
is  from  where  do  the  checklists  come.  It  seems  reasonau'e  that  they  could  come 
from  either  the  expertise  or  naturalistic  oriented  approaches. 

Adversarv-oriented  Approach 

Adversary  oriented  approaches  to  evaluation  attempt  to  minimize  the  bias  inherent  in 
all  evaluations  by  incorporating  planned  opposition  .  By  generating  opposing 
points  of  view  around  a  given  issue  it  is  hoped  thai  a  clear  course  of  action  will 
emerge.  Fairness  and  balance  are  the  driving  forces  behind  these  kinds  of 
evaluations.  Typically  adversarial  evaluations  attempt  to  bring  out  the  pros  and 
cons  of  an  issue;  however,  some  kinds  of  evaluations,  such  as  committee  hearings, 
deliberately  consider  several  viewpoints.  Adversarial  approaches,  potentially,  have 
much  in  common  with  other  evaluation  approaches  such  as  expertise-oriented 
approaches  because  they  rely  heavily  on  the  use  ot  expert  witnesses. 
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An  adversarial  form  of  problem  clarification  would  be  especially  useful  where  two 
or  more  strongly  competing  opinions  exist.  Perhaps  the  strongest  asset  ot  an 
adversarial  form  of  instructional  analysis  would  be  a  recognition  that  different  kinds 
of  problems  can  be  justified.  It  may  also  provide  a  forum  for  the  discussion  ot 
multiple  perspectives  on  the  value  systems  of  key  players  in  the  development 
process. 

Summary  and  Future  Research 

Clearly  the  emphasis  in  this  paper  has  been  upon  building  a  case  for  new 
approaches  to  needs  assessment.  While  some  practical  suggestions  have  been  made 
as  to  how  a  needs  assessment  may  be  patterned  after  a  particular  evaluation 
approach,  more  work  needs  to  be  done  to  make  these  suggestions  more  concrete. 
Certainly,  data  collection  and  data  analysis  procedures  used  in  naturalistic  kinds  ot 
inquiry  such  as  those  carried  out  by  anthropologists  should  be  considered  by 
developers  during  needs  assessments.  Also,  future  work  should  attempt  to  assess 
the  use  of  these  new  approaches  in  actual  development  projects. 
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Background 

One  weakness 

has  been  J-.e  difficulty  that,  many  prote 


One  weakness  of  the  fteld  of  educational  ledinology,  like  many  academic  fields  of  study, 

any  professionals  and  practitioners  have  had  explaining  Hie 


aeiimuun  m  via  «Apirical  studies,  flutitw  evident  uiat  previous       ,  .. 
least  partialis-  responsible  for  th«  differing  viewpoints  with  which  we  are  faced,  and  are 
partly  responsible  for  the  difficulty  that  we  encounter  when  trying  to  explain  the 
terminology  of  the  field  Foremost  among  these  difficult  terms  is  educational  technology. 

There  have  been  manv  attempts  ta  define  the  term  "educational  technology"  (AECT,  1977). 
These  have  proven  to  be  unsatisfactory  in  the  long  term,  as  witnesseo  by  thefrequeni. 
efforts  of  the  Association  for  Educational  Coauoupfcatfrns  and  Technology  s  Definition 
and  Terminology  Committee.  Because  of  me  many  changes  ;n  both  concept  and  practice 
that  occur  within  the  field,  the  definition  must  undergo  change.  It  r&ny  be  more  useful  to 
consider  using  sonw  different  techniques  cf  conceptual  analysis  when  investigating  the 
term  educational  technology  . 

Rather  than  focus  on  one  particular  definition,  this  inquiry  is  intended  to  study  multiple 
conceptions  of  the  term  "educational  technology' .  Several  writers i  have .discussed  the  idea 
of  multiple  conceptions  of  educational  technology  (cf.  Saettler,  1990  and  Roundtree.1979). 
These  studies  have  treated  multiple  conceptions  of  educational  technology  in  a 
linear/historical  fashion.arguing  that  one  conception  is  replaced  by  another  as  tune  passes. 
However,  this  paper  argues  that  multiple  conceptions  of  the  term  educational  technology 
exist  simultaneously. 

Purpose 

The  purpose  of  this  paper  is  to  analyze  the  tern  "educational  technology"  and  to 
demonstrate  that  multiple  conceptions  of  it  do,  in  fact,  exist  simultaneously.  This  study  has 
four  major  objectives:  (1)  to  present  the  importance  of  conceptual  analysis  tor 
investigations  in  the  field  of  educational  technology.  (2)  io  demonstrate  that  the  use  ot 
•met  .phof  is  appropriate  for  investigating  the  concept  of  educational  technology,  (3)  to 
provide  a  theoretical  framework  for  the  choice  of  the  metaphors:  art,  craft,  science,  and 
engineering  as  ways  of  viewing  educational  technology,  (4)  to  provide  instances  of  each  of 
the  four  metaphors,  art,  science,  craft,  and  engineering  from  the  literature  of  the  field  ot 
educational  technology, 

Philosophical  Basis  and  Conceptual  Analysis 

Concepts  regulate  and  direct  our  thought.  But  concepts  are  not  just  matters  of  the  intellect. 
They  govern  the  way  in  which  we  function  in  our  daily  lives.  "Our  concepts  structure  what 
we  perceive,  how  we  get  around  in  the  world,  and  how  we  relate  to  other  people  (Lakoff 
and  Johnson,  1980  p.3).  This  means  that  the  concepts  that  we  use,  and  the  conceptual 
systems  which  with  we  think,  are  at  the  heart  of  how  we  define  our  everyday  realities. 

For  purposes  of  this  study  it  means  that  the  way  in  which  an  individual  holds  a  particular 
concept,  in  this  case  the  concept  of  educational  technology,  will  help  to  determine  not  only 
how  that  individual  views  the  field  of  educational  technology  but  also  how  that  concept 
guides  the  individual's  actions  within  the  field  of  educational  technology. 
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Since  the  conceptions  that  one  holds  of  educational  technology  can  help  determine 
professionals  view  the  field  and  how  professionals  function  within  the  field,  an  analysis .  ut 
how  different  individuals  view  the  field  will  contribute  to  a  greater  understanding  ot  the 
field  of  educational  technology.  A  better  understanding  of  the  concept  of  e<  ucatxona 
technology  should  result  in  the  identification  of  some  of  the  assumptions  and 
presuppositions  that  help  drive  the  activities  of  the  field  of  educational  technology. 

The  questions;  "Why  do  educational  technologists  think  and  act  the  way  that  they  dor  and 
"What  might  this  thought  and  activity  mean  for  the  field  of  educational  technology  ?  1  n< 
first  question  is  primarily  a  historical  question,  the  second  is  a  matter  of  reflection.  These 
ideas  are  intertwined  and  separating  them  is  difficult.  As  such,  both  should  be  considered 
in  any  conceptual  study. 

Historical  interest  in  concepts 

Any  conceptual  study  will  be  historical  in  some  sense  for  two  reasons;  first,  we  can  find 
c  .tain  general  historical  patterns  that  often  appear  when  studying  the  development  ot 
concepts.  Many  concepts  pass  through  two  periods:  one  where  there j  seems ;  to  be  in  strong 
agreement,  followed  by  one  where  exceptions  begin  to  appe^  (Fleck,  1979).  An  exarriple 
of  this  in  our  field  is  the  changes  that  have  occurred  1 ..  the  definitions  of  the  term 
"educational  technology". 

A  stylistic  bond  exists  between  many  concepts  of  an  historical  period.  This  is  based  on  the 
influence  that  these  concepts  have  upon  each  other.  There  is  "a  thought  style  which 
determines  the  formulation  of  a  concept"  (Fleck,  1979  p.9).  Perhaps  it  is  more  accurate  to 
sav  that  there  are  different  'thought  styles'  that  co-exist,  and  that  each  thought  style  has 
some  impact  on  how  a  given  concept  is  viewed  Different  thought  styles  emerge  more 
dominant  at  different  points  in  time.  The  co-existence  of  different  thought  styles  might 
explain  how  concepts  can  be  thought  of  differently  by  different  groups  of  people. 
Historical  investigation  shows  that  strong  disagreements  often  occur  about  the  definition  ot 
concepts  (Fleck,  1979).  Conflicts  exist  in  spite  of  the  fact  that,  from  a  logical  viewpoint, 
the  various  conceptions  may  be  equally  plausible. 

For  our  field,  it  means  that  educational  technologists  are  not  at  complete  liberty  to  replace  a 
"mass-media"  concept  of  educational  technology  with  one  based  solely  on  the  idea  ot  the 
systems  approach.  There  are  'thought  styles'  that  exist  that  would  oppose  this  attempt. 
These  'thought  styles'  may  exist  within  the  field  of  educational  technology,  or  they  may 
exist  outside  of  the  field. 

Reflective  interests  ejmcejag 

In  addition  to  historical,  interest  in  concepts,  there  is  also  reflective  interest.  "Concepts  in 
education  are  often  used  in  ways  in  which  they  are  normative,  as  well  as  descriptive 
"(Gusfield  1975  p.xvii).  In  this  sense,  they  can  be  characterized  like  theories,  which  have 
also  been  classified  as  being  normative  or  descriptive.  Studying  concepts  can  help  us  to 
determine  the  significance  of  events  and  processes  as  well  as  just  describing  them. 
Studying  concepts  can  also  reveal  the  subtle  aspects,  that  is,  the  emphasis  which  may  be 
placed  (  n  certain  words  and  actions. 
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A  concept  might  be  considered  as  a  sort  of  instrument,  something  that  serves  us.  But  it  can 
also  be  thought  of  as  something  that  we  are  inclined  to  serve,  either  voluntary  or  not.  The 
ideas  that  matter  most  to  people  are  the  ones  that  give  their  experience  direction  and 
meaning.  But  frequently  this  is  done  at  a  subconscious  level.  Lakoff  and  Johnson, 
describe  this,  saying, 

But  our  conceptual  system  is  not  something  that  we  are  normally  aware  of. 
In  most  of  the  little  things  we  do  every  day,  we  simply  think  and  act  more 
or  less  automatically  along  certain  lines.  Just  what  these  lines  are  is  by  no 
means  obvious.  One  way  to  find  out  is  by  looking  at  language.  Since 
communication  is  based  on  the  same  conceptual  system  that  we  use  in 
thinking  and  acting,  language  is  an  important  source  of  evidence  for  what 
that  system  is  like  (1980,  p.  3). 

The  concept  of  educational  technology  is  interpreted  in  many  ways.  The  problem  of  trying 
to  analyze  complex  concepts  is  that  there  is  not  one  singular  meaning  which  can  be  arrived 
at  This  difficulty  can  be  thought  of  as  a  choice  between  clarity  and  utility.  It  would  be 
simpler  if  one  could  define  the  concept  and  point  to  what  it  conveys.  This  would  represent 
a  consistent,  uniform  meaning,  and  imply  that  consensus  exists  among  educational 
technologists.  But  this  would  be  misleading  and  inaccurate.  It  would  not  serve  those  who 
wish  to  study  the  concept  of  educational  technology  to  grasp  the  different  usages  and 
contexts  of  that  concept.  Nor  would  it  help  them  to  understand  the  reasons  why  these 
differences  arose.  And  it  would  not  aid  them  in  their  professional  need  to  cope  with  the 
ambiguity  which  is  so  prevalent  in  the  field  of  educational  technology. 

Approaches  to  conceptual  study 

There  are  a  number  of  different  ways  in  which  a  phrase  or  term  can  been  studied.  These 
approaches  can  be  classified  in  three  broad  categories,  those  that  are  historical  in  nature, 
those  that  are  associated  with  sociology,  and  those  having  to  do  with  linguistics  and 
philosophy. 

Historical  approaches 

Etymology  focuses  on  the  history  of  the  term.  Here  the  meaning  of  the  term  is  traced  back 
to  its  earliest  roots.  Then  subsequent  definitions  and  uses  of  the  term  are  provided.  It  is 
sometimes  argued  that  since  a  term  held  a  certain  meaning  in  the  past,  that  this  past 
meaning,  or  portions  of  it,  can  be  carried  forward  and  reintroduced,  and  used  as  it  had  been 
in  the  past. 

Studies  of  the  history  of  ideas  also  describe  the  changes  in  the  meanings  of  particular  terms 
over  time.  As  originally  conceived,  this  approach  focused  on  the  "root  ideas'  which 
constituted  a  broader  concept  (Lovejoy,  1940).  Often,  these  root  ideas  were  the  conditions, 
necessary  and  sufficient,  that  set  the  boundaries  of  a  particular  concept.  It  was  reasoned 
that  if  the  essential  components  of  a  concept  could  be  understood,  then  the  meaning  ot  a 
concept  would  become  clear. 
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Sociological  approaches 

Max  Weber  introduced  the  "ideal-type"  as  a  way  to  investigate  complex  concepts.  Weber 
characterizes  this  as  an  imagined  construction, 

...not  a  description  of  reality  but  it  aims  to  give  unambiguous  means  of 
expression  to  such  a  description  (Weber,  1963  p.28). 

Gusfield  (1975)  uses  another  sociological  approach  called  the  "paired  opposites".  The  idea 
is  to  study  a  concept  by  relating  it  to  its  opposite.  His  study  is  a  discussion  of  the  concepts 
of  community  and  society.  He  provides  two  poles  between  which  real  instances  can  be 
ranged  as  teing  more  or  less  of  one  type  or  another.  Gusfield  claims  that  "in  using  type  as 
paired  opposites,  the  sociologist  heightens  the  understanding  of  each"(p  13). 

In  the  phiiOSophicaJ/analytic  approach,  writers  expose  the  problems  inherent  in  the  most 
fashionable  definitions  of  a  given  concept,  they  identify  and  criticize  possible  alternatives, 
and  then  put  forth  their  own  viewpoint.  There  are  a  number  of  variations  on  this  theme. 

The  'dictiosiarv  approach'  which  is,  "the  determination  of  current  usage  from  the  users  of 
special  vocabularies"  (Churchill,  1968  p.4).  There  have  been  many  attempts  to  explain 
educational  technology  by  "defining"  it  (AECT.1977).  These  attempts  were  not  what 
might  be  commonly  referred  to  as  definitions,  relating  the  meaning  of  a  term  in  a  sentence 
or  two.  Rather  they  were  often  lengthy,  deductive  treatments  of  the  author's  conception  of 
educational  technology.  It  is  important  to  note  that  these  studies  were  deductive.  By  this  it 
is  meant  that,  the  author  started  from  a  single  statement  about  v/hat  educational  technology 
was  and  then  proceeded  to  elaborate  on  that  statement,  about  what  the  parts  of  the  definition 
were  and  how  they  fit  together.  This  was  called  the  "complete"  definition  statement  of 
educational  technology.  It  was  a  deductive,  and  systematic  explanation  of  the  term 
educational  technology. 

These  definition  statements  share  some  commonalties  with  what  many  philosophers  and 
social  scientists  refer  to  as  conceptual  analysis,  Both  are  systematic  and  detailed  treatments 
of  complex  terms.  However,  as  Gowin  (1982)  explains: 

Concept  analysis,  however,  is  different  from  simply  giving  definitions  of 
the  meaning  of  terms.  We  would  not  complete  an  analysis  by  looking  in 
dictionaries  for  the  meaning  of  words.  Instead,  we  are  seeking  a  reason  or 
explanation  for  why  we  use  the  terms  in.  particular  ways  ([.200). 

This  is  the  key  to  the  so-called  "Oxford  School  of  Philosophy"  or  the  "Philosophy  of 
Ordinary  Language".  His  from  this  area  of  study  that  we  have  received  the  famous  dictum 
'die  meaning  is  in  the  use', 

Another  very  popular  method  of  conceptual  analysis  concerns  itself  with  determining  the 
necessary  and  sufficient  conditions  for  a  particular  instance  to  be  part  of  a  larger 
classification,  In  this  approach,  the  attributes  of  a  particular  concept  that  are  determined  to 
be  necessary  in  order  for  its  existence  are  established.  Then,  certain  of  these  attributes,  or 
combinations  of  attributes,  are  deemed  to  be  sufficient  for  an  instance  to  be  considered  a 
meml-r  of  a  larger  set  or  category  (Lakoff  and  Johnson,  1980  and  others).  This  k  often 
considered  to  be  the  'traditional  approach'  to  analyzing  concepts. 
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Still  another  approach  to  understanding  the  meaning  of  a  concept  is  through  the  study  of 
metaphors.  A  metaphor,  Egon  Guba  writes, 

...is  a  figure  of  speech  in  which  the  meaning  of  a  term  or  phrase  is 
transferred  from  the  object  it  ordinarily  designates  to  another  object  so  as  to 
provide  new  insight  or  perspective  on  the  latter.  In  a  broader  sense,  the  term 
metaphor  may  also  designate  a  process  whereby  the  meanings  and 
relationships  of  one  theory  or  model  may  be  used  to  suggest  meanings  or 
relationships  n  another  arena  (Guba,1978  as  cited  in  Smith,  1982  p.52). 

Studying  Mela  ^iors 

Studying  metaphors  is  recognized  as  important  to  the  study  of  literature.  Lakoff  and 
Johnson  (1980'.  identify  other  focal  points  for  metaphorical  study; 

Me'  phor  is  for  most  people  a  device  of  the  poetic  imagination  and  rhetorical 
fi'.'ji'ih-a  matter  of  extraordinary  rather  than  ordinary  language.  Moreover, 
'd  jtsphor  is  typically  viewed  as  characteristic  of  language  alone,  a  matter  of 
voids  rather  than  thought  or  action  (p.  3). 

There  are  many  different  views  of  metaphor.  There  are  the  substitution,  semantic  and 
comparison  views.  There  is  the  interactionist  theory  of  metaphor.  There  is  the  pragmatic 
approach  to  metaphor.  And  there  is  the  literal  vs  root  metaphor  controversy.  These  have 
been  classified  along  a  constructivist-nonconstructivist  line  (Ortony,1979).  In  the 
construe tivi st  tradition,  Lakoff  and  Johnson  (1980)  discuss  a  variety  of  "root  metaphors" 
for  analyzine  conceptual  structures.  One  such  example  is  the  "argument  is  war"  metaphor; 

Your  claims  are  indefensible 

He  attacked  every  weak  point  in  my  argument 

his  criticisms  were  right  on  target 

I  demolished  his  argument 

I've  mver  won  an  argument  with  him 

You  disagree?  Okay  shoot 

If  you  use  that  strategy,  he'll  wipe  you  out 

He  shot  down  all  of  my  arguments. 

(Lakoff  and  Johnson,1980  p.4) 

Driscoll  (1990)  describes  the  usefulness  of  using  the  constructivist  approach  to  metaphor  in 
educational  technology,  especially  within  the  area  of  instructional  design.  Driscoll  suggests 
that  using  metaphor  is  a  way  to  identify  the  assumptions  on  which  instructional  design  is 
based. 

But  one  must  use  caution  when  employing  metaphors.  One  may  select  good  ones  or  poor 
ones  and  it  is  often  difficult  to  know  the  difference  without  proper  criteria  (Smith,  1981). 
Further,  when  used  without  proper  consideration,  they  can  be  misleading  and  inaccurate 
(Schon,1979). 

The  use  of  metaphor  to  investigate  educational  technology  is  not  unique  to  this  study  (cf. 
Hlynka  &  Nelson,  1991  and  Driscoll  1990).  Hlynka  and  Nelson  (1991)  have  discussed 
the  metaphors  of  the  term  "educational  technology".  They  constructed  these  "root 
metaphors"  by  identifying  patterns  of  language  used  in  the  literature  of  the  professional 
literature. 
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But  there  is  another  aspect  to  metaphor.  It  is  the  figurative  side.  It  is  a  more  traditional 
approach  to  the  study  of  metaphor.  It  involves  the  overt  characterization  of  educational 
technology.  Sometimes  these  characterizations  are  conscious  decisions  and  sometimes  they 
are  not.  Regardless  of  the  intent  of  the  intent  these  decisions,  these  characterizations  have  a 
theoretical  bases  that  is  historically  rich. 

If  care  is  taken  'metaphorical  study'  is  a  viable  form  of  conceptual  study  for  investigating 
educational  technology.  It  is  one  way  of  getting  at  the  concepts  and  assumptions  that  exist 
within  the  reality  of  professional  practice.  The  purpose  of  "exposing  metaphors,  and  the 
corresponding  myths  from  which  they're  derived,  frees  us  to  consider  alternative  ones" 
(Driscoll,  1990  p.33)  and  thereby  laying  the  groundwork  to  change  practice. 

It  is  the  study  of  metaphor  in  terms  of  "common  language"  and  its  relation  to  thought  and 
action  which  is  central  to  this  study.  Here,  the  investigation  of  metaphor  will  focus  on 
characterizations  of  the  practice  of  educational  technology. 

On  technology 

Most  people  involved  in  the  field  of  education  are  familiar  with  the  concepts  of  "education" 
and  "technology"  long  before  they  realize  that  the  two  concepts  together  make  up  a  specific 
field  of  study.  When  combined,  these  two  words  take  on  a  new  meaning,  one  which  is 
multifaceted  and  dynamic. 

Considering  the  many  ways  in  which  "educational  technology"  has  been  conceptualized, 
the  question  arises,"which  ones  would  be  most  helpful  in  the  study  of  educational 
technology?"  A  historical  overview  might  be  helpful. 

Many  of  the  studies  that  are  conceptual  treatments  of  "educational  technology"  include  a 
definition  or  a  discussion  of  the  term  technology'.  Far  fewer  of  these  studies  include  a 
definition  or  discussion  of  the  terms  'education'  or  'educational'.  One  could  infer  several 
things  from  this:  (1)  that  the  word  education  is  understood,  (2)  that  technology  is  the 
dominant  or  more  important  word  in  the  phrase  educational  technology,  (3)  that  the  word 
'educational'  in  the  phrase  educational  technology  is  a  descriptor  modifying  the  purpose  of 
the  technology,  such  as  in  'educational  television'. 

Two  of  the  most  frequently  cited  definitions  of  technology  that  are  included  in  conceptual 
treatments  of  educational  technology  are: 

"Technology  means  the  systematic  application  of  scientific  or  other 
organized  knowledge  to  practical  tasks"  (Galbraith,1967  p.24  as  cited  in 
AECT,  1977,  p.57;  Silber,1972,  p.36  and  Heinich,Molenda,and  Russell, 
1989,  p.24). 

"Technology  includes  processes,  systems,  management  and  control 
mechanisms,  both  human  and  non-human,  and  above  all...  a  way  of 
looking  at  problems  as  to  their  interest,  and  difficulty,  the  feasibility  of 
technical  solutions,  and  the  economic  values-broadly  considered-of  those 
solutions."  (Finn,  in  Eboch,  1963,  p  17  as  cited  in  AECT,  1977,  p.57  and 
more  completely  in  Ely,  1970,  p.  85) 
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A  problem  with  the  first  definition  is  the  attempt  to  separate  the  types  of  knowledge  that  are 
identified.  Is  "scientific  knowledge"  not  "organized  knowledge"?  Practically  speaking,  the 
definition  of  technology  would  read:  Technology  is  the  systematic  application  of  organized 
knowledge  to  practical  tasks'.  Perkins  (1986)  argues  that,  in  fact,  it  is  the  design  or 
organization  of  information  which  makes  it  knowledge.  If  this  is  true  than  "technology  is 
the  application  of  knowledge  to  practical  tasks."  To  follow  this  line  of  reasoning  one 
would  have  to  ask  "what  would  one  apply  to  practical  tasks  if  not  knowledge?  The 
answer  is  not  cleai. 

The  second  definition,  while  certainly  less  succinct  than  the  first,  is  important  because  it 
identifies  many  of  the  component  characteristics  of  technology.  However,  this  particular 
definition  has  a  context  or  interest  attached  to  it.  Finn  was  a  proponent  of  making  the  field  a 
profession  and  changing  the  name  of  the  field  to  include  the  word  technology.  It  is  not  a 
coincidence  that  he  defined  technology  to  include  many  of  the  functions  which  he  believed 
to  be  important  to  professionalizing  the  audiovisual  field  (Finn,1953).  This  suggests  that 
this  definition  of  technology  is  "loaded"  and  not  appropriate  for  use  in  this  study. 

In  a  lengthy  conceptual  study  of  educational  technology  W.  Kenneth  Richmond  (1970) 
examines  definitions  of  words  that  he  argues  are  the  historical  basis  of  the  term 
"technology".  He  includes  the  following: 

Technical-Belonging  to  arts,  not  in  common  popular  usage. 

(John  Walker,  A  Critical  Pronouncing  Dictionary  of  the  English  Language, 

New  Edition,  carefully  revised,  corrected  and  enlarged,  Nelson,  1845.) 

Technical-Of  or  in  or  peculiar  to  a  particular  art  or  science  or  craft  (t.  terms, 
uKill);  of  or  in  or  the  mechanical  arts  (t.education,  school),  technicality- 
being  t.,a  t.term:  technics -doctrine  of  the  arts,  technique;  technique-manner 
of  artistic  execution,  the  part  of  artistic  work  that  is  reducible  to  formula, 
mechanical  skill  in  art.[Gk  tekhne  art] 

(H.W.Fowler  and  F.  G.  Fowler  compiled,  The  Pocket  Oxford  Dictionary 
of  Current  English,  Third  Edition,  Oxford  at  the  Clarendon  Press,  1939) 

Technic-1.=  foll.2.=  technique  (usu.pl)  doctrine  of  arts  in  general:  (pi.) 
technical  terms,  details,  methods.etc.  Hence  technician-person  skilled  in  the 
technique  of  a  particular  art,  or  in  arts  generally. [Gk  tekhnikos  (tekhne  art,)] 

Technical-Of  or  in  a  particular  art,  science,  handicraft,  etc.,  as  terms,  skill, 
difficulty;  of,  for  in,  the  mechanical  arts,  as  education,  school;  legally  such, 
in  the  eyes  of  the  law,  as  assault. 

Technique-Mode  of  artistic  execution  in  music,  painting,  etc:  mechanical 
skill  in  art.[F,as  technic] 

Technology-Science  of  the  industrial  arts;  ethnological  study  of 
development  of  arts.[Gk  tekhnologia  (tekhne  art  -logy)] 
(H.  W.  Fowler  and  F.  G.  Fowler  (eds.),  Concise  Oxford  Dictionary  of 
Current  English,  Fourth  Edition,  Revised  by  E.  Mcintosh,  Oxford  at 
Clarendon  Press,  1951) 
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Richmond  points  out  that  these  are  related  terms  and  that  the  list  of  these  related  terms 
grown  tenfold  in  the  period  from  1845  to  1951.  An  undated  listing  of  these  terms,  from 
Oxford  English  Dictionary  (1989)  includes: 

technic-of  or  pertaining  to  art.or  craft 
A  l.Pertainingtoartoranart; 

2.  Skillfully  made  or  constructed  . 

B.  1  .A  technical  term,  expression,  point,  or  detail;  a  technicality 

2.  a  Technical  details  or  methods  collectively;  the  technical  department  of  a 
subject;  the  formal  or  mechanical  part  of  art. 

b.  Collective  pi.  technics  in  same  sense;  also  construed  as  a  singular. 

3.  The  science  or  study  of  art  or  arts,  esp.  of  mechanical  or  industrial  arts 

technical-A.l  Of  a  person;  Skilled  in  or  practically  conversant  with  some 
particular  art  or  subject. 

2.  Of  a  thing ;  Skillfully  done  or  made 

3.  a.  Belonging  or  relating  to  an  art  or  arts;  appropriate  or  peculiar  to, 
characteristic  of,  a  particular  art,  science,  profession,  or  occupation;  also,  of 
or  pertaining  to  the  mechanical  arts  and  applied  sciences  generally. 

b.  said  of  words,  terms,  phrases,  etc.,  or  their  senses  of  acceptations 

c.  Of  an  author,  a  treatise,  etc:  Using  technical  terms:  treating  a  subject 
technically.  . 

d.  Technically  so  called  or  regarded;  that  is  such  as  a  technical  point  of  view 

4.  Of,  pertaining  to,  or  designating  a  market  in  which  prices  are  determined 
chiefly  by  internal  factors 

Technically-In  a  technical  manner;  in  relation  to  the  arts  and  applied 
sciences,  or  to  a  practical  art  or  subject;  according  to  technical  methods. 

Technician-a.  A  person  conversant  with  the  technicalities  of  a  particular 
subject,  b.  One  skilled  in  the  technique  or  mechanical  part  of  an  art,  as 
music,  or  painting,  c  .  A  person  qualified  in  the  practical  application  of  one 
of  the  sciences  or  mechanical  arts;  now  esp.  a  person  whose  job  is  to  carry 
out  practical  work  in  a  laboratory  or  to  give  assistance  with  technical 
equipment. 

Technicism-1.  A  technical  term  or  expression,  a  technicality. 

2.  Technical  quality  or  character;  a  condition  in  which  practical  results  or 

methods  are  stressed. 

Technicize-To  make  technical;  to  subject  to  a  high  degree  of  technicality 

Technification-The  adoption  or  imposition  of  technical  methods. 

technique-a.  Manner  of  artistic  expression  execution  or  performance  in 
relation  to  formal  or  practical  details  (as  distinct  from  general  effect, 
expression,  sentiment,  etc.);  the  mechanical  or  formal  part  of  an  art,  esp.  of 
any  of  the  fine  arts;  the  manner  of  execution  or  performance  in  any 
discipline,  profession,  or  sport;  also,  skill  or  ability  in  this  department  of 
one's  art;  mechanical  skill  in  artistic  or  technical  work,  loosely  a  skilful  or 
efficient  means  of  achieving  a  purpose;  a  characteristic  way  of  proceeding;  a 
knack. 
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Technological- 1.  Belonging  to  technical  phraseology  or  methods. 

2.  Relating  to  or  dealing  with  the  study  of  the  arts,  esp.  the  industrial  arts. 

3.  Pertaining  to  or  characterized  by  technology;  resulting  from 
developments  in  technology. 

Technology-systematic  treatment  l.a.  A  discourse  or  treatise  on  an  art  or 
arts;  the  scientific  study  of  the  practical  or  industrial  arts.  b.  Practical  arts 
collectively,  c.  A  particular  practical  or  industrial  art. 

(Oxford  English  Dictionary, 1989) 

The  definitions  of  words  related  to  technology,  like  the  definitions  of  educational 
technology,  include  many  words  and  phrases  that  are  repeated,  or  rather,  held  constant. 
Some  of  these  repeated  words  and  phrases  are:  "art",  "artistic  execution",  "craft", 
"systematic  study",  "scientific  study",  "science",  "mechanical  skill",  "method", 
"mechanical  arts",  "industrial  arts",  "practical  arts",  and  "applied  science". 

The  last  four  of  these,  "mechanical  arts",  "industrial  arts",  "practical  arts",  and  "applied 
science"  are  all  associated  with  'engineering'.  The  remaining  words  and  phrases  may  all  be 
subsumed  by  the  words  "art",  "science",  and  "craft". 

There  are  four  views  of  technology  that  come  close  to  representing  it  as  art,  science,  craft, 
nnd  engineering.  These  essays  were  produced  by  Martin  Heiddegger  (1977),  Jacques  Ellul 
(1963),  Victor  Ferkis  (1969),  and  William  Barrett  (1978).  Heiddegger  (1977)  writes  for  a 
technology  that  is  art; 

Technology  is  therefore  no  mere  means.  Technology  is  a  way  of  revealing. 
If  we  give  heed  this,  then  another  whole  realm  for  the  essence  of  technology 
will  open  itself  up  to  us.  It  is  the  realm  of  revealing,  i.e., of  truth. 

This  prospect  strikes  us  as  strange.  Indeed,  it  should  do  so,  should  dc  so 
as  persistently  as  possible  and  with  so  much  urgency  that  we  will  finally 
take  seriously  the  simple  question  of  what  "technology"  means.  The  word 
stems  from  the  Greek.  Technikon  means  that  which  belongs  to  techne.  We 
must  observe  two  things  with  respect  to  the  meaning  of  the  word.  One  is 
that  techne  is  the  name  not  only  for  the  activities  and  skills  of  a  craftsman, 
but  also  for  the  arts  of  the  mind  and  the  fine  arts.  Techne  belongs  to 
bringing-forth,  to  poiesis;  it  is  something  poietic  (Heidegger,1977  p.  12- 


Writing  for  a  technology  that  may  be  characterized  as  more  craft  oriented  Ellul  (1963) 
focuses  on  the  word  technique.  He  writes: 

In  fact,  technique  is  nothing  more  than  means  and  the  ensemble  of 
means. ..Techniques  considered  as  methods  of  operation  present  certain 
common  characteristics  and  certain  general  tendencies... The  technical 
operation  includes  every  operation  carried  out  in  accordance  with  a  certain 
method  in  order  to  attain  a  particular  end  (Ellul,1963  p.19). 
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Ferkis  (1969)  criticizes  Ellul's  notion  of  technique  and  writes  for  a  technology  that  is  more 
an  engineering  concept  argues: 

In  so  broadly  conceiving  technology  there  is  obviously  the  danger  of  falling 
into  the  trap  of  Ellul  and  other  writers  for  whom  technology  is  technique 
and  everything  is  technique  and  therefore  all  human  actions  are 
technological.  We  can  avoid  this  pitfall  if  we  think  of  technology  as  a  self- 
conscious  o^anized  means  of  affecting  the  physical  or  social  environment, 
capable  of  being  objectified  and  transmitted  to  others,  and  effective  largely 
independently  of  the  subjective  dispositions  or  personal  talents  of  those 
involved.  Thus  an  opera  singer  exhibits  "technique"  but  is  not  really 
utilizing  a  technology. (Ferkis,  1969  p.37-38) 

"Technology  is  embodied  technique"  (Barrett,  1978  p.  18).  A  technique  is  a  standard 
method  that  can  be  taught.  Continuing  his  discussion  of  the  inseparability  of  technology 
and  science  Barrett  argues  that: 

It  is  method,  and  'he  dominance  of  method  that  marks  the  chief  difference 
between  modem  science  and  the  science  of  the  Greeks.  It  is  an  attempt  to 
control  and  predict  phenomena  rather  than  submit  to  its  existence  that 
differentiates  between  the  two.  Modern  science  could  be  considered  as  a 
movement  from  theory  into  practice  with  method  at  the  core  of  the 
discussion.  The  individual  (knower)  no  longer  merely  observes  what  is. 
Rather,  the  individual  sets  the  conditions,  asks  the  questions  and  elicits  the 
answers.  The  meanings  of  the  questions  and  answers  themselves  become 
less  separable  in  the  conditions  established  by  the  new  science.  Technology 
is  not  the  application  of  the  new  science  it  is  the  heart  of  the  new  science 
(Barrett,1978  p.). 

An  analysis  of  these  definitions  and  conceptual  discussions  reveals  that  technology  is  seen 
in  different  ways  but  there  is  a  common  theme  that  runs  through  them.  The  theme  is  that 
technology  is  a  process. 

Art,  Craft,  Science  and  Engineering  in  Education 

A  study  of  the  history  of  education  over  the  last  one  hundred  years  reveals  that  education, 
and  many  of  the  concepts  that  are  associated  with  education,  are  often  characterized  as  art, 
science,  craft,  and  enginee-ing.  Philosopher  John  Dewey  has,  to  some  degree, 
characterized  education  in  ail  four  of  these  veins. 

Among  his  many  and  varied  writings  Dewey  examines  education  as  an  art  (Dewey,  1928 
1929a)  as  a  science  (Dewey,  1928,  1929b),  and  as  Engineering  (Dewey,  1929c).  In  a 
discussion  of  the  relationship  of  art.  science,  and  engineering  to  education  Dewey  (1929b) 
writes: 

That,  in  concrete  operation,  education  is  an  art,  either  a  mechanical  art  or  a 
fine  art  is  unquestionable.  If  there  were  an  opposition  between  science  and 
art,  I  ^ould  be  compelled  to  side  with  those  who  assert  that  education  is  an 
art.  But  there  is  no  opposition,  although  there  is  a  distinction.  We  must  not 
be  misled  hy  words.  Engineering  is,  in  actual  practice,  an  art.  But  it  is  an 
art  that  progressively  incorporates  more  and  more  science  into  itself,  more 
of  mathematics,  physics  and  chemistry.  It  is  the  kind  of  art  it  is  precisely 
because  of  a  content  of  scientific  subject-matter  which  guides  it  as  a  practical 
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operation.  There  is  room  for  the  original  and  daring  projects  of  exceptional 
individuals.  But  their  distinction  lies  not  in  the  fact  that  they  turn  their  back 
on  science,  but  in  the  fact  that  they  make  new  integrations  of  scientific 
material  and  turn  it  to  nt  w  and  previously  unfamiliar  and  unforeseen  uses. 
When,  in  education,  the  psychologist  or  observer  and  experimentalist  in  any 
field  reduces  his  findings  to  a  rule  which  is  uniformly  adopted,  then,  only, 
is  there  a  result  which  is  objectionable  and  destructive  of  the  free  play  of 
education  as  an  art. 

But  this  happens  not  because  of  scientific  method  but  because  of  departure 
from  it.  It  is  not  the  capable  engineer  who  treats  scientific  findings  as 
imposing  upon  him  a  certain  course  which  is  to  be  rigidly  adhered  to:  it  is 
the  third-  or  fourth-rate  man  who  adopts  this  course.  Even  more,  it  is  the 
unskilled  day  laborer  who  follows  it.  For  even  if  the  practice  adopted  is  one 
nat  follows  from  science  and  could  not  have  been  discovered  or  employed 
except  for  science,  when  it  is  converted  into  a  uniform  rule  of  procedure  it 
becomes  an  empirical  rule-of-thumb  procedure-just  as  a  person  may  use  a 
table  of  logarithms  mechanically  without  knowing  anything  about 
mathematics  (Dewey,  1929,  p.635-636). 

It  must  be  acknowledged  that  in  the  early  part  of  the  twentieth  century  'science'  often  held  a 
meaning  that  is  substantially  different  then  the  meaning  that  is  ascribed  to  it  today. 
However,  because  of  the  influence  that  Dewey  has  had  on  the  field  of  education,  later 
scholars  adopted  his  language  in  new  contexts.  This  adoption  of  language  without 
accounting  for  changes  in  popular  meaning  has  led  to  confusion  within  the  field  of 
education  in  general,  and  also  in  the  field  of  educational  technology. 

Contemporaries  of  Dewey  have  also  used  the  wo/ds  art,,  science  and  engineering  to  discuss 
education  and  concepts  that  are  related  to  it.  William  Heard  Kilpatrick  (1925)  argues  that 
education  is  an  art.  E.  L.  Thorndike  (1913)  promotes  a  view  that  education  is  a  science. 
Franklin  Bobbin  refers  to  "educational  engineers"  (1912). 

Individuals  writing  after  the  era  of  progressive  education  had  come  to  a  close,  continued 
discussing  education  and  some  of  its  related  concepts  as  .art,  science,  and  engineering. 
Gilbert  Highet  (1958)  referred  to  teaching  as  an  art.  W.W.  Charters  (1945)  discusses 
educational  engineering.  B.F.  Skinner  (1954)  argues  for  a  science  of  learning. 

The  idea  of  craft  is  noticeably  missing  from  the  previous  discussion.  Perhaps  this  was  due 
to  the  growing  acceptance  of  "scientific  -method"  during  the  early  portion  of  the  twentieth 
century.  Science,  as  a  modern  concept,  would  have  been  seen  in  opposition  to  craft  which 
may  have  been  taken  as  outmoded. 

It  is  in  the  era  of  post-progressive  education  that  the  word  craft  gains  popularity  in  the 
education  literature  (ci.  Thorn,  Blum  berg).  This  may  be  due  to  a  backlash  against  scientific 
and  engineering  approaches  to  education.  It  may  also  be  due  to  the  acceptance  of  certain 
contributions  of  science  to  the  field  of  education.  The  use  of  scientific  knowledge  in  the 
field  of  education  has  become  more  refined.  It  has  become  clearer  that  science  contributes 
to  the  development  of  educational  procedures  and  not  to  the  setting  of  educational  goals 
(KiiebardJ986)  One  could  infer  from  this  that  scientific  based  procedures  alone  are 
insufficient  for  completing  the  educational  enterprise.  As  such,  the  idea  of  craft,  because  it 
can  include  these  scientific  procedures,  gains  in  popularity. 
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The  idea  that,  both,  technology  and  education  has  been  viewed  as  art,  scienr*  »n*meenag, 
and  craft  has  been  with  us  for  for  a  long  time.  Therefore,  it  seems  ap;  «  these 

four  words  or  interpretations  of  technology  and  education:  art,  i»v  rait,  ana 
engineering,  will  be  the  interpretations  that  are  used  to  analyze  educations  technology  in 
the  remainder  of  this  study. 

Art,  craft,  science  and  engineering  in  educational  technology 

The  words  art,  craft,  science,  and  engineering  are  not  new  as  descriptors  of  educational 
technology.  Educational  technology  has  been  characterized  as  being  an  art  by  Hylenka 
(1991),  Bass  and  Dills  (1984),  and  Davies  (1984).  It  has  been  called  a  craft .by  Finn 
(1953)  andDriscoll  (1990)  and  a  science  by  Lumsdaine  (1964)  and  Reigeluth  (1983).  It 
has  also  been  called  engineering  by  Bern  (1961),  Szabo  (1968),  andHeimch  (i;984). 

The  words  art,  craft,  science  and  engineering  are  really  quite  familiar  to  the  literature  of 
educational  technology.  But  an  analysis  of  these  familiar  and  obvious  connotations  shows 
that  they  are  not  easily  distinguishable  from  one  another.  And  seldom  are  the  reasons  for 
using  these  characterizations  and  their  implications  made  explicit  by  their  users  in  our  field. 

Science 

The  characterization  of  educational  technology  as  a  science  is  over  thirty  years  old.  In  an 
article  relating  technology  to  education  Finn  (1960)  chose  not  to  distinguish  between 
science  and  technology. 

For  purposes  of  this  paper,  no  distinction  is  made  between  "science"  and 
"technology"  or  between  "pure"  and  "applied"  science  (Finn,  1960  p.84). 

A  discussion  of  the  distinction  between  science  and  technology  would  certainly  have  been 
helpful  to  Finn's  readers,  but  it  was  not  central  to  the  thesis  of  the  paper.  But  by  not 
making  the  distinction  Finn  contributes  to  the  confusion  and  ambiguity  of  the  concept. 

LumsHaine  (1964)  outlines  the  premises  and  importance  of  a  science  of  instruction.  There 
are  turee  ideas  that  are  central  to  the  existence  of  a  science  of  instruction:  1)  the  desire  to 
generate  rules  which  can  be  generalized  beyond  specific  situations,  2)  the  generating  and 
testing  of  hypotheses,  3)  the  use  of  experimental  techniques.  Lumsdaine  writes: 

The  attempt  to  develop  instructional  materials  not  only  to  serve  as  a  vehicle 
for  testing  hypotheses  but  also  to  attain  a  high  degree  of  instructional 
effectiveness  tends  to  generate  provisional  "rules"  for  effective 
programming  of  instruction,  which  can  be  implemented  in  experimental 
materials  even  though  these  rules  are  not  tested  directly  within  the  confines 
of  a  given  programming  project.  Such  working  guidelines  need  to  be 
translated  into  hypotheses  that  can  be  subjected  to  experimental  test  if  a 
genuine  science  of  instruction  is  to  evolve  (Lumsdaine,  1964  p.400). 

He  concludes: 

Finally,  it  may  well  be  re-emphasized  that  the  differences  in  present 
conceptions  and  forms  of  programming  as  well  as  their  current  theoretical 
rationales,  are  less  important  than  the  basic  conviction  that  instruction  is 
amenable  to  systematic  description  and  improvement  through  experimental 
inquiry  (Lumsdaine,  1964  p.401). 
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The  science  conception  of  educational  technology  holds  that  it  is  an  area  of  study 
(instruction)  which  should  be  investigated  by  the  use  of  methods  and  techniques 
(experimental)  associated  with  the  traditional  sciences.  The  goal  is  to  be  able  to  prescribe 
interventions  (actions)  to  bring  about  desired  outcomes. 

Engineering 

The  idea  of  problem  solving  is  crucial  to  an  engineering  conception  of  educational 
technology.  Discussing  the  method  of  the  educational  engineer,  W.  W.  Charters  (1945) 
writes: 

First,  the  educational  engineer  accepts  an  idea  to  develop,  a  problem  to 
solve,  or  a  question  to  answer.. ..Having  accepted  an  idea  to  work  upon,  his 
next  step  is  a  logical  definition  of  the  problem....He  must  know  precisely 
what  he  is  supposed  to  do.  When  a  problem  has  been  defined,  the 
educational  engineer  analyzes  it  to  discover  the  factors  that  must  be 
considered.  Each  of  the  factors  requires  analysis  that  must  be  considered. 
Thereupon,  the  assignment  is  planned.  His  attention  is  engrossed  in  the 
methods  to  be  used  to  control  the  factors.  He  concentrates  upon  the  "how" 
of  his  project;  he  decides  that  he  will  define  the  objectives  in  this  manner, 
make  his  selection  of  words  in  that  fashion,  and  determine  the  related 
characteristics  in  another  way.  These  specifications  he  formuiates  with  the 
greatest  of  care  because  upon  their  excellence  rests  the  success  of  his 
project. 

Having  completed  his  plans,  he  proceeds  to  construct  his  project  by 
carrying  out  the  operations  which  have  been  specified  in  the  manner  decided 
upon.  The  labor  of  construction  consumes  the  major  portion  of  the  time 
spent  upon  a  project  although  their  plan  is  the  most  critical  step  in  program 
building.  When  the  instrument  has  been  constructed,  operation  follows. 
Planning  and  constriction  have  been  carried  on  to  that  end.  The  idea  has 
been  given  body  on  the  assumption  that,  when  properly  implemented,  it  will 
improve  instruction.  The  problem  has  been  solved  with  the  expectation  that 
its  solution  will  increase  the  efficiency  of  schorl  processes. 

The  final  phase  of  the  engineering  method  in  education  is  evaluation.  When 
the  operation  is  completed,  it  is  tested  to  see  if  it  fulfills  its  function 
adequately.  If  it  does,  it  continues  to  be  used.  If  it  does  not,  it  may  be 
discarded.  But  if  the  idea  is  properly  Seated  in  the  early  stages  of  the 
engineering  process,  evaluation  and  testing  result  in  further  improvements 
in  the  program  rather  than  its  rejection  (Charters,  1945,  p.36-37  &  56). 

Heinich  (1984)  accepts  the  premise  that  educational  technology  is  basically  an  engineering 
view.  He  argues  that  educational  technology  is  an  applied  field  which  is  concerned  with 
means  and  ends. 

Instruction  is  the  management  of  learning,  and  instructional  management, 
like  engineering,  is  a  diss  of  its  own  made  up  of  a  complex  organization  of 
men,  machines,  and  processes  (p.84). 
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But,  Heinich  also  challenges  die  scientific  conception  of  educational  technology. 

A  large-scale  instructional  problem  may  not  be  best  analyzed  in  terms  of 
individual  personalogical  variables  or  isolated  media  attributes  but  by  a 
consideration  of  demographics  of  students,  organizational  relationships,  the 
sociology  of  the  instructional  environment,  delivery  systems,  etc.  The 
laboratory  approach  of  many  of  our  researchers  frequently  is  not  compatible 
with  instructional  management  realities. 

We  need  to  remember  that  research  techniques  designed  to  establish  cause- 
and-effect  relationships  may  not  be  suitable  for  means-ands  problems. 
Research  designs  and  statistical  techniques  most  appropriate  for  conclusion- 
oriented  research  may  impose  artificial  and  unrealistic  constraints  on 
decision-oriented  (situation-specific)  questions. ...The  techniques  of 
operations  research  in  engineering  and  business  administration  are  more 
"sympatico"  with  decision-oriented  problems  in  instructional  systems. 

We  have  emulated  science  in  our  research  for  too  long.  There  are  many 
reasons  for  this,  a  few  implied  earlier,  but  surely  the  higher  status  accorded 
science  over  technology  in  higher  education  and  the  virtual  adoration  of 
SCIENCE  by  the  public  are  important  factors  (Heinich,  1984,  p.84). 

Critics  of  educational  technology  often  treat  the  engineering  and  the  science  conceptions  of 
educational  technology  as  if  they  were  the  same.  No  doubt  there  are  some  similarities,  and 
there  are  many  cases  where  such  a  categorization  scheme  is  justified.  But  certainly  an 
engineering  conception  of  educational  technology  differs  substantially  from  a  science 
conception.  The  science  conception  focuses  on  prescriptions  derived  from  experimental 
methods,  the  engineering  conception  on  problem  solving  and  applications  that  include  more 
considerations  than  the  experimental  method. 

Art-then  Craft 

Davies  (1991)  dissects  the  concepts  of  craft  and  art  as  they  relate  to  instructional 
development. 

Admitting  the  role  of  art,  craft,  and  science  in  instructional  development, 
however,  is  not  the  same  as  suggesting  that  ID,  itself,  is  a  craft  or  science. 
Indeed,  instructional  development  is  very  definitely  an  art.  There  have 
been,  however,  many  attempts  to  make  it  a  craft,  while  arguing  it  is  a 
science.  It  is  ironic  that  when  developers  highlight  the  design, 
development,  implementation  and  evaluation  st^es-in  the  belief  that  these 
somehow  confer  the  status  of  scientific  endeavor-they  are,  in  fact, 
reinforcing  the  craft  side  of  what  is  essentially  a  creative  act  of  inquiry. 

Attempts  to  make  instructional  development  a  craft  or  a  science  have 
supplied  in  the  first  case  a  heuristic,  and  in  the  latter  case  a  recipe  or 
algorithm,  that  has  largely  failed  to  realize  the  potential  of  ID.  To  a  certain 
extent,  the  problem  arises  from  a  misunderstanding  of  the  nature  of  art, 
craft,  and  science.  So  has  the  equation  of  science  with  technology.... 

Art  and  craft  are  often  confused.  They  are  usually  seen  as  somewhat 
similar,  the  one  shading  the  other.  Science,  on  the  other  hand,  is  perceived 
as  an  entirely  different  process,  probably  because  of  its  strong  empirical 


3u0 

37K 


15 


basis.  The  distinctions,  however,  in  reality  are  much  more  subtle  than  they 
appear.  It  is  easier  to  distinguish  between  art  and  cratt  (DaviesJ.991  p.96). 

The  distinctions  between  art  and  craft  Collingwood  (1938)  argues  in 
Principles  of  Art  are  best  seen  from  the  perspective  of  certain  key 
characteristics.  Theyinc  le: 

•  Distinctions  bet  woen  means  and  ends. 

•  Distinctions  between  planning  and  implementation. 

•  Distinctions  between  raw  material  and  finished 
product. 

•  Distinctions  between  form  and  matter. 

In  essence,  if  the  distinctions  are  maintained  in  an  activity,  Collingwood 
argues  that  a  craft  is  involved.  If  the  distinctions  are  largely  absent,  blur,  or 
coexist  with  the  other,  then  an  art  is  at  work  (Davies,  1991,  pp.97 -98). 

Later  Davies  discusses  the  criteria  for  successful  instructional  development.  They  are 
phrased  in  the  form  of  questions. 

Are  the  end  results  (sensitivities,  values,  attitudes,  skills,  knowledge) 

worthwhile?...  .  ,  . 

Have  the  understandings,  attitudes  and  skills  essential  to  mastery  been 

Have  foe  learnings  been  integrated  into  people's  lives,  so  that  reliable 
performance  is  achieved?.. .(Davies,  1991  pJ  02-103). 

Davies  argues,  with  a  blend  of  passion  and  reason,  that  instructional  development  should 
be  treated  as  an  art.  He  differentiates  the  concepts  of  systematic  and  systemic.  Systematic 
is  associated  with  craft  and  systemic  with  art.  The  crux  of  the  argument  is  thai  systematic 
infers  separate  stages  in  a  process,  that  means  and  ends  are  separate.  Systemic  infers  that 
there  are  not  necessarily  separate  stages  in  instructional  development,  that  they  can  co-exist. 

There  is  some  confusion  in  this  overall  argument.  One  the  one  hand  Davies  states: 
"Indeed  instructional  development  is  very  definitely  an  art  (p.96).  This  can  betaKcn  to 
mean  that  if  one  is  'doing'  instructional  development  one  must  be  engaged  in  art. 
However  the  subsequent  discussion  and  reasoning  about  the  systematic-systemic 
difference  leaves  the  impression  that  in  juctaal  development  can  be  either  an  art  or  a 
craft. 

An  implication 

What  does  this  mean  for  the  field  of  educational  technology?  Here,  the  discussion  will  be 
limited  to  implications  for  professionalism.  Finn  (1953)  argues  that 

a  profession  has,  at  least,  toe  characteristics:  (a)  an  intellectual 
technique,  (b)  an  application  of  that  technique  to  the  practical  affa^s  of  man, 

(c)  a  period  of  long  training  necessary  before  entering  into  the  profession, 

(d)  an  association  of  the  members  of  the  profession  into  a  closely-knit  group 
with  a  high  quality  of  communication  between  members,  (e)  a  series  of 
standards  and  a  statement  of  ethics  which  is  enforced,  and  (f)  an  organized 
body  of  intellectual  theory  constantly  expanding  by  research  (Finn,  195 3 
p.7). 
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Finn  then  explains  the  importance  of  the  sixth  criteria: 

...the  most  fundamental  and  most  important  characteristic  of  a  profession  is 
that  the  skills  involved  are  founded  upon  a  body  of  intellectual  theory  and 
research.  Furthermore,  this  systematic  theory  is  constantly  being  expanded 
by  research  and  thinking  within  the  profession.  As  Whitehead  says,  "...the 
practice  of  a  profession  cannot  be  disjoined  from  its  theoretical 
understanding  or  vice  versa.. .The  antithesis  to  a  profession  is  an  avocation 
based  upon  customary  activities  and  modified  by  the  trial  and  error  of 
practice.  Such  an  avocation  is  a  Craft...."  (16:557)  The  difference  between 
a  bricklayer  and  the  architect  lies  right  here  (Finn,1953  p.8). 

Finn's  intent  was  to  move  the  audio- visual  field  from  a  status  of  lowly  craft  to  that  of  a 
profession.  The  field  of  educational  technology  has  long  labored  under  Finn's  influence  in 
regards  to  professionalism.  The  Association  for  Educational  Communications  and 
Technology  (AECT,  1977)  has  reiterated  these  conditions  for  being  a  profession. 

But  does  this  explanation  hold  up  in  its  entirety?  Another  way  to  view  "the  trial  and  error 
of  practice"  is  to  call  it  evaluation.  Evaluation,  or  some  like  term  (criteria  check,  testing),  is 
a  concept  that  is  key  to  all  of  the  conceptions  of  educational  technology  that  have  been 
discussed.  If  this  is  the  lone  criteria  for  craft  status,  then  educational  technology  is  a  craft. 
Educational  technologists  should  be  pleased  to  be  thought  of  as  doing  a  craft,  but  many  are 
not.  Likely  this  is  due  to  historical  considerations.  As  stated  above,  craft  was  seen  as 
something  below  science,  art,  and  engineering. 

One  possible  way  around  this  difficulty  is  to  consider  Pratte  and  Rury's  (1991)  discussion 
of  professionalism.  They  argue  that  "professionalism.. .is  an  ideal  to  which  individuals  and 
occupational  groups  aspire,  in  order  to  distinguish  themselves  from  other  workers  (p.60)." 
Pratte  and  Rury  focus  on  teaching,  they  call  it  a  craft-profession.  They  claim 

...a  craft-profession  does  not  rest  on  a  highly  formal  or  codified  body  of 
knowledge.  Instead,  competence  for  craft-professionals  is  defined  in  terms 
of  various  skills  and  practices,  reflecting  a  different  sort  of  knowledge 
base... much  of  their  knowledge  is  embodied,  something  that  they  learn  by 
doing  and  that  is  experientially  learned,  rather  than  acquired  in  a  systematic, 
highly  formal  fashion  (p.62). 

To  a  large  extent  the  distinction  between  the  conceptual  (formal)  knowledge 
characteristic  of  the  expert  professions  and  the  embodied  (experiential) 
knowledge  of  the  crafts  is  too  finely  drawn,  perhaps  even  artificial,  as  all 
occupations  require  both  formal  and  experiential  knowledge.  It  appears  that 
the  fundamental  point  is  one  of  identifying  the  extent  to  which  a  given 
occupation  is  dominated  by  one  or  the  other  (p.63). 

the  concept  of  craft  seems  to  belong  to  both  the  worid  of  skill  and  the  world 
of  calling  or  profession.  The  ordinary  language  appeal  of  the  term  is  to 
some  shared  understanding  of  what  a  craft  is,  to  a  union  of  technique  and 
theory.  One  can,  we  suppose,  be  too  scrupulous  about  the  matter,  but  the 
concept  of  craft  should  be  understood  as  an  art  or  skill  in  a  field  or  calling 
having  appropriate  standards-in  short,  action  willingly  engaged  in  accord 
with  publiciy  agreed  on  criteria  (p.63). 
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Certainly  language  can  be  'overly  scrutinized'.  But  it  seems  that  discussions  of 
educational  technology  as  Art,  Craft,  Science  and  Engineering  do  shed  light  on  how  users 
of  these  terms  view  educational  technology  as  a  field  of  study  and  as  a  profession.  It 
seems  that  sometimes  the  use  of  these  terms  is  conscious  and  sometimes  it  is  not. 
Sometimes  it  is  an  attempt  to  gain  prestige,  sometimes  it  is  an  attempt  to  avoid  using  a 
word,  and  sometimes  it  is  simply  an  attempt  to  communicate  an  idea. 

Conclusion 

It  appears  as  if  there  are  different  ways  in  which  educational  technology  can  be 
characterized.  At  first  glance,  one  might  think  that  all  design  is  science,  development  is 
craft,  and  all  management  functions  are  engineering.  But,  the  reality  is  that  this  view  is 
simplistic.  It  is  much  more  complicated  than  that.  What  the  analysis  in  this  paper  seems  to 
show  is  that  the  four  characterizations  of  the  term  'educational  technology*  do  currently 
exist  in  our  field  and  they  have  been  present  for  some  time. 
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INTRODUCTION 

"...studies  show  that  even  though  teachers  have 
a  high  level  of  media  competence  and  equipment  is 
available  to  them  in  their  buildings .  the  majority  of 
teachers  do  not  make  extensive  use  of  educational 
media."  (p.  60) 

C.  Edward  Streeter 

Perhaps  one  of  the  reasons  teachers  do  not  use  media 
extensively  is  due  to  the  lack  of  instruction  relative  to  effective 
use  as  it  relates  to  specific  disciplines.  Knowing  about  media  or 
knowing  that  it  exists  is  not  the  same  as  knowing  how  to  apply  it 
to  enhance  the  instructional  needs  of  a  particular  discipline. 
Experienced  or  inservice  teachers  at  the  secondary  level  have  an 
understanding  of  what  works  well  for  them.  The  intent  of  this 
study  was  to  draw  upon  that  experience  by  having  inservice 
teachers  recommend  instructional  media  competencies  that  are 
beneficial  to  them  in  their  disciplines. 

Proctor  (1983),  reviewed  the  available  literature  relating 
to  media  use  and  found  a  difference  between  pedagogical  theory 
and  classroom  practice.  "The  prescriptive  literature,  based 
largely  on  the  results  of  empirical  studies,  outlines  the  benefits 
attributable  to  the  use  of  media;  but  the  descriptive  literature, 
based  largely  on  the  results  of  surveys  and  questionnaires, 
reveals  one  almost  universal  theme:  MEDIA  ARE  SELDOM 
USED.M(p  5j  An  analysis  of  several  media  research  reviews  such 

as  Lumsdaine  (1963),  Saettler  (1968),  Levie  and  Dickie  (1973), 
and  Wilkinson  (1980)  supports  the  differences  between 
research  conducted  to  determine  potential  contribution  to  the 
teaching/learning  process  and  those  which  identify  or 
determine  the  current  conditions  of  use  within  the  classroom. 

One  application  of  the  descriptive  type  research  is 
significant  in  relation  to  the  Teacher  Education  Programs  in  the 
Schools  and  Colleges  of  Education.  Starting  as  early  as  1932, 
attempts  have  been  made  to  identify  what  is  being  taught  in  the 
media  courses  offered  in  Teacher  Education  Programs.  Stracke 
(1932),  Starnes  (1937-38),  Taylor  (1942),  de  Kieffer  (1948. 
1959,  1970,  1977),  Meierhenry  (1966),  Rome  (1973), 
McCutcheon  (1984)  and  others  have  attempted  to  provide  an 
understanding  of  curriculum  content  of  media  courses.  Some 
like  Stracke  (1932),  Starnes  (1937-38),  de  Kieffer  (1948.  1959, 
1970,  1977),  and  McCutcheon  (1984)  have  reviewed  the  course 
content  of  media  courses  within  the  schools  and  colleges  of 
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education.  While  others,  Taylor  (1942),  deBernardis  and  Brown 
(1946),  Fulton  and  White  (1959),  Rome  (1973),  Lare  (1974), 
Jones  (1982),  etc.,  have  considered  media  skills  or 
competencies  in  use  or  recommended  by  inservice  education 
personnel  including  classroom  teachers,  administrators  and 
college  instructors  for  methods  courses  as  well  as  instructional 
media  faculty.  These  studies  make  suggestions,  either  implicit 
or  implied,  regarding  instructional  media  course  content  based 
on  feedback  obtained  from  users. 

Reconciling  the  descriptive  and  prescriptive  research 
has  long  been  a  problem  as  evidenced  by  Fulton  (1960): 
"Research  evidence  indicates  that  we  know  much  more  about 
what  we  should  be  doing  with  modern  communicative  media  in 
education  than  we  are  actually  doing."(p  496)  Due  to  these 

apparent  contradictions  in  research  findings,  drawing 
conclusions  can  sometimes  be  frustrating,  Allen  (1973)  points 
out  some  factors  that  contribute  to  this  frustration. 

"...a  look  back  over  the  past  50  years  of  research  is 
both  encouraging  and  discouraging.  We  can  see  no 
neatly  organized  body  of  research  findings  that  can  be 
used  to  guide  our  practice.  For  the  most  part,  past 
research  has  been  haphazard,  poorly  integrated,  and 
lacking  any  theoretical  structure.  We  see  little 
evidence  that  what  we  have  found  out  is  being  applied 
to  instruction  or  that  we  are  even  asking  the  right 
questions. "(p  49) 

Allen  (1973)  seems  to  suggest  a  direction  for  future  research 
when  he  says:  The  major  problem  facing  the  researcher  is  the 
determination  of  the  specific  conditions  under  which  different 
media  should  be  employed,  how  the  media  should  be  designed, 
and  with  what  kinds  of  learners. n(p  ^ 

PURPOSE  OF  THE  STUDY 

This  study  was  an  investigation  to  determine  the 
instructional  media  competencies  that  inservice  teachers  of 
secondary  education  teaching  disciplines  recommend  for  pre- 
service  teachers  in  their  discipline.  The  primary  purpose  of  this 
study  was  to  identify  the  instructional  media  competencies 
common  to  all  teaching  disciplines.  In  addition,  the  media 
competencies  unique  to  each  discipline  were  identified. 
Consideration  was  also  given  to  the  perceived  value  of 
instructional  media  use  in  the  classroom,  length  of  tenure  as  a 
teacher  and  the  teaching  location,  namely  Hawaii,  Oregon  and 
Utah.  Recommendations  were  studied  relative  to  the 
instructional  approach  in  relation  to  teaching  discipline. 
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DESIGN  AND  PROCEDURES 

Hypotheses 

The  results  of  this  study  determine  the  retention  or 
rejection  of  the  following  null  hypotheses: 

Kq  1.  There  is  no  significant  difference  in  the 

recommendation  of  instructional  media  competencies 
among  teachers  in  the  secondary  education  teaching 
disciplines. 

H0  2.  There  is  no  significant  difference  in  the 

perceived  value  of  instructional  media  use  in  the 
classroom  among  teachers  in  secondary  education 
teaching  disciplines. 

Hq  3.  There  is  no  significant  difference  in  the 

recommended  instructional  approach  among  teachers 
in  secondary  education  teaching  disciplines. 

Research  Questions 

The  study  gave  additional  consideration  to  the  following 
research  questions  each  related  to  the  instructional  media 
competency  recommendations  and  teaching  disciplines. 

1.  Is  there  a  difference  in  instructional  media 
competency  recommendations  by  teachers  in  secondary 
education  teaching  disciplines  based  on  the  state  in  which 
they  are  teaching? 

2.  Is  there  any  difference  in  the  recommendations  of 
instructional  media  competencies  among  all  teachers 

in  Hawaii,  Oregon  or  Utah? 

3.  Is  there  a  difference  in  the  perceived  value  of 
instructional  media  use  in  the  classroom  among  teachers 
in  secondary  education  teaching  disciplines  based  on  the 
state  in  which  they  are  teaching? 

4.  Is  there  any  difference  in  instructional  media 
competency  recommendations  based  on  perceived  value 
of  media  use  in  the  classroom? 

5.  Is  there  any  difference  in  instructional  media 
competency  recommendations  by  teachers  in  secondary 


3  S  f  ; 
360 


education  teaching  disciplines  based  on  perceived  value 
of  media  use  in  the  classroom? 

6.  Is  there  any  difference  in  instructional  media 
competency  recommendations  by  teachers  in  secondary 
education  teaching  disciplines  based  on  years  of 
teaching  experience? 

7.  Is  there  any  difference  in  the  factors  for  media  non- 
use  among  teachers  in  secondary  education  teaching 
disciplines? 


This  paper  does  not  summarize  the  entire  study. 
Only  the  findings  for  hypotheses  HqI,  Hq2  and 

research  questions  1-5  above  are  presented  here. 
These  specifically  relate  to  the  recommendations  of 
instructional  media  competencies  by  inscrvice 
teachers  from  selected  secondary  education 
disciplines  and  their  perceived  value  of  media  use  in 
the  classroom. 


The  instrument  was  a  mail  administered  questionnaire.  It 
consisted  of  instructional  media  competencies  combined  with  a 
six  point  continuous  rating  scale  ranging  from  ngX 
recommended  to  recommended.  This  allowed  the  respondent 
to  judgmentally  recommend  the  inclusion  of  each  competency  in 
a  pre-servlce  teacher  education  program. 

The  development  of  the  instrument  was  accomplished  in 
three  stages: 

1.  Related  literature;  (Stracke  (1932),  deBernardis 
and  Brown  (1946),  Jensen  (1986),  Meierhenry  (1966),  Okoboji 
(1959)  and  Streeter  (1969);  were  studied  to  identify  lists  of 
media  competencies  commonly  accepted  in  the  field.  Current 
course  syllabi  from  selected  universities  and  colleges  offering 
introductory  instructional  media  courses  and  current 
instructional  media  text  books  were  also  surveyed  to  refine  and 
finalize  the  list  of  competencies.  (See  Table  1) 

2.  A  jury  panel  of  experts  was  selected  that  consisted 
of  instructional  media  specialists  and  instructors  as  well  as 
research  specialists  with  expertise  in  questionnaire 
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development.  Using  a  modified  Delphi  procedure,  the  panel 
members  were  asked  to  respond  to  the  list  of  competencies  on 
the  questionnaire  as  well  as  its  format  and  design. 

3.  The  final  questionnaire  (see  Appendix  A)  was 
reviewed  by  a  research  consultant  for  final  recommendations. 
The  questionnaire  was  then  submitted  to  the  panel  for  final 
acceptance  or  rejection.  No  panel  members  responded  with 
further  suggestions. 

The  Statistical  Design 

The  design  of  the  study  included  the  following: 

1.  The  population  consisted  of  in-service  teachers  of 
secondary  education  teaching  disciplines  from  public  schools  in 
the  states  of  Hawaii,  Oregon  and  Utah.  A  random  numbers  list 
was  computer  generated.  Using  current  state  school  directories 
twenty-five  (25)  schools  were  randomly  selected  from  each 
state.  Each  list  of  schools  was  renewed  for  a  match  of  size  and 
location,  ie;,  rural,  urban  and  metropolitan.  A  mailing  list  was 
computer  generated  for  each  of  thirteen  disciplines  for  each 
school.  A  total  of  three-hundred  and  twenty-five  (325)  mailings 
for  each  state  or  a  total  of  nine-hundred  and  seventy-five  (975) 
was  prepared. 

Using  the  procedures  outlined  by  DiUman  (1978)  a 
questionnaire  was  mailed  to  teachers  of  the  secondary  education 
teaching  disciplines.  A  cover  letter  was  included  explaining  the 
purpose  of  the  study  and  the  importance  of  each  response  to  the 
success  of  the  study.  Each  questionnaire  was  numbered  for 
follow-up  purposes.  A  business  reply  envelope  was  included  for 
return  of  the  questionnaire.  All  the  questionnaires  were  mailed 
on  the  same  day. 

Following  Dillman's  (1978)  procedure  a  post-card  follow- 
up  reminder  was  prepared  and  mailed  to  all  teachers  in  the 
study  exactly  one  week  after  the  first  mailing. 

As  the  questionnaires  were  returned  they  were  checked 
off  against  the  original  mailing  list.  The  intent  was  to  send  a 
second  mailer  with  questionnaire  to  those  not  responding 
within  a  three  week  period.  This  was  not  accomplished  due  to 
the  lack  of  time  before  the  end  of  public  school  in  the  three 
states  of  the  study. 

Originally  there  were  thirteen  (13)  secondary  education 
teaching  disciplines  identified.  As  the  questionnaires  were 
returned  and  checked  off  it  was  noticed  that  there  were  very 
few  being  returned  from  "Computer  Science*  teachers.  Most  of 
the  ones  that  were  identified  as  "Computer  Science"  teachers 
had  been  addressed  to  "Mathematics"  teachers.  It  was  also 
observed  that  many  being  returned  from  "Mathematics" 
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teachers  had  actually  been  addressed  to  "Computer  Science" 
teachers.  It  was  decided  that  for  purposes  of  this  study  it  would 
be  appropriate  to  combine  the  teaching  discipline  category 
"Computer  Science"  with  "Mathematics".   The  combined 
category  was  entitled  "Mathematics/  Computer  Science."  This 
resulted  in  twelve  secondary  education  teaching  disciplines  for 
the  study. 

A  period  of  one  week  was  allowed  to  lapse  after  receiving 
what  seemed  to  be  the  last  response  before  the  data  were 
compiled  and  submitted  for  computer  analysis.  At  the  time  of 
data  compilation  four -hundred  and  sixteen  (416)  or  43%  of  the 
total  questionnaires  had  been  returned.  All  questionnaires  were 
usaole  and  were  included  in  the  data  pool.  Of  the  four-hundred 
and  sixteen  (416)  returns  one-hundred  and  fifty- three  (153) 
were  from  Hawaii.  This  represents  a  47%  return  rate.  There 
were  one-hundred  and  twenty- six  (126)  from  Oregon  for  a  39% 
return.  Utah  had  one-hundred  and  thirty-seven  (137)  or  a  42% 
return.  The  numbers  returned  were  good  considering  only  one 
follow-up  reminder  was  sent.  There  were  four  additional 
questionnaires  returned  after  the  data  were  compiled  and 
submitted.  They  were  left  out  of  the  study.  (See  Table  2  for 
response  frequencies  by  teaching  discipline  and  location.) 

2.  The  respondents  were  asked  to  make 
recommendations  for  instructional  media  competencies, 
perceived  value  of  instructional  media  use  in  the  classroom  and 
recommendations  of  an  instructional  approach.  The  responses 
were  recorded  using  a  six  point  continuous  scale,  with  values 
ranging  from  a  low  of  0  to  a  high  of  5.0. 

The  data  from  the  questionnaires  were  transferred  to 
computer  by  the  writer.  The  raw  data  were  stored  in  disk  form 
and  then  transferred  to  the  university  main  frame  Prime 
Computer  for  analysis.  The  analysis  flics  were  created  and 
computed  using  the  Statistical  Package  for  Social  Sciences-X 
(SPSSX)  release  3.0. 

3.  The  one-way  analysis  of  variance  was  selected  as  the 
statistical  analysis  to  test  the  three  null  hypotheses  because  they 
require  the  contrasting  of  two  (2)  or  more  means.  The  Duncan 
Multiple  Range  Test  was  utilized  to  determine  which  of  the 
means  of  the  teaching  disciplines  differed  significantly  for  each 
of  the  instructional  media  competencies. 

In  addition  to  the  one-way  analyses  of  variance  as  described 
above,  the  t-Test  was  also  used  to  develop  information  regarding 
one  of  the  research  questions  (number  6)  described  above. 

A  total  of  six-hundred  and  eighty  (680)  one-way  ANOVA's 
were  computed.  A  total  of  six- hundred  and  seventy- two  t- test's 
were  completed.  A  series  of  cross  tabulation  tables  and  mean's 
tables  were  also  completed. 
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4.  The  F  Test  was  used  to  test  the  significance  of  the 
analysis.  The  alpha  level  for  significance  was  set  at  p=.05  for  all 
one-way  ANOVA's.  For  informational  purposes  the  t-Test  results 
were  compiled  for  p=.10  as  well  as  p=.05. 

5.  Utilization  of  the  data  and  information  compiled 
requires  more  than  quantitative  analysis.  The  objective  of  the 
study  is  to  develop  a  series  of  competency  lists  that  are,  first 
common  to  all  secondary  education  teaching  disciplines  and  that 
are,  second,  unique  to  each  teaching  discipline  or  groups  of 
disciplines.  This  requires  reviewing  the  data  and  making 
qualitative  judgments  about  the  importance  of  each  competency 
by  teaching  discipline. 


DATA  ANALYSIS  AND  FINDINGS 

Responses  to  the  questionnaire  elicited  from  teachers  in 
secondary  education  teaching  disciplines  within  selected  states 
were  compiled  for  computer  analysis  using  the  SPSSX  release 
3.0  statistical  package.  The  computer  analysis  produced  the 
following  findings  for  each  of  the  hypotheses  and  research 
questions  listed  below. 

Hypotheses 

Hq  1.  There  is  no  significant  difference  in  the 

recommendation  of  instructional  media  competencies 
among  teachers  in  the  secondary  education  teaching 
disciplines. 

A  one-way  analysis  of  variance  was  computed  for  ea^h  of 
fifty-six  (56)  instructional  media  competencies  plus  four  (4) 
"other"  categories.   (The  four  "other"  categories  produced  no 
responses  on  the  returns  so  they  were  disregarded  in  the 
analysis.)  Of  the  fifty-six  ANOVA's  completed,  thirty-six  (36)  or 
64%  indicated  a  significant  difference  in  recommendations  of 
instructional  media  competencies  between  teachers  in  the 
secondary  education  teaching  disciplines  at  the  .05  level  of 
confidence  or  above.  For  these  competencies  null  hypothesis 
H0  1  is  rejected.  Twenty  (20)  or  36%  indicated  no  significant 

difference.  For  these  competencies  null  hypothesis  Hq  1  is 

retained. 

(Tables  are  available  upon  request.) 
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H0  2.  There  is  no  significant  difference  in  the  perceived 

value  of  instructional  media  use  in  the  classroom 
among  teachers  in  secondary  education  teaching 
disciplines. 

A  one-way  analysis  of  variance  was  computed  along  with 
the  Duncan  Multiple  Range  Test.  A  significant  difference  was 
indicated  among  how  the  teaching  disciplines  perceive  the  value 
of  instructional  media  use  in  the  classroom.  Null  hypothesis  Hq 

2  is  rejected  at  the  .0002  level  of  confidence.  The  results  of  the 
Duncan  Multiple  Range  Test  indicate  that  Industrial  Arts 
teachers  have  the  highest  perceived  value  of  instructional  media 
use  in  the  classroom  with  a  mean  of  4.17  while  Musio  teachers 
with  a  mean  of  2.90  have  the  lowest  perceived  value  of 
instructional  media  use  in  the  classroom.  Eleven  of  the  teaching 
disciplines  (Industrial  Arts.  Home  Economics,  Health,  Social 
Sciences,  Sciences,  Foreign  Languages,  Language  Arts,  Business, 
Art,  Physical  Education  and  Mathematics/Computer  Science) 
were  significantly  different  from  Music.  Three  of  the  teaching 
disciplines  (Industrial  Arts,  Home  Economics  and  Health)  were 
significantly  different  than  Mathematics/Computer  Science  and 
Music.  Table  3  displays  the  relative  ranking  of  teaching 
disciplines  by  perceived  value  of  instructional  media  use  in  the 
classroom. 

(Tables  are  available  upon  request.) 


Research  Questions 

While  not  stated  as  formal  hypothesis  the  following 
research  questions  have  been  included  in  order  to  help  clarify 
the  information  developed  by  this  research.  Each  of  the 
questions  presented  below  expands  upon  what  has  been 
discussed  relative  to  the  impact  of  secondary  education  teaching 
disciplines  upon  teachers  recommendations  for  specific 
instructional  media  competencies  and  their  perceived  value  of 
instructional  media  use  in  the  classroom. 

1.  Is  there  a  difference  in  instructional  media 
competency  recommendations  by  teachers  in 
secondary  education  teaching  disciplines  based  on  the 
state  in  which  they  are  teaching? 

A  one-way  analysis  of  variance  was  computed  for  each  of 
the  fifty-six  instructional  media  competencies  controlling  for  the 
states  of  Hawaii,  Oregon  and  Utah.  A  total  of  one-hundred  and 
sixty-eight  (168)  ANOVA's  were  completed. 
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For  the  state  of  Hawaii,  eleven  (11)  or  20%  of  the 
recommendations  for  instructional  media  competencies  tested 
significantly  different  among  secondary  education  teaching 
disciplines  at  the  .05  level  of  confidence  or  higher.  Forty-five 
(45)  or  80%  of  the  instructional  media  competencies  showed  no 
significant  difference  among  teaching  disciplines. 

For  the  state  of  Oregon,  twenty-two  (22)  or  39%  of  the 
recommendations  for  instructional  media  competencies  tested 
significantly  different  among  secondary  education  teaching 
disciplines  at  the  .05  level  of  confidence  or  higher.  Thirty- four 
(34)  or  61%  of  the  instructional  media  competencies  showed  no 
significant  difference  among  teaching  disciplines. 

For  the  state  of  Utah,  eleven  (1 1)  or  20%  of  the 
recommendations  for  instructional  media  competencies  tested 
significantly  different  among  secondary  education  teaching 
disciplines  at  the  .05  level  of  confidence  or  higher.  Forty-five 
(45)  or  80%  of  the  instructional  media  competencies  showed  no 
significant  difference  among  teaching  disciplines. 

A  Duncan  Multiple  Range  Test  was  computed  for  each 
instructional  media  competency  found  to  have  significant 
difference  in  order  to  determine  which  teaching  disciplines 
were  different. 

(Tables  are  available  upon  request.) 

2.  Is  there  any  difference  in  instructional  media 
competency  recommendations  among  all  teachers  in 
Hawaii,  Oregon  or  Utah? 

A  one-way  analysis  of  variance  was  computed  for  each  of 
the  fifty-six  instructional  media  competencies  using  teaching 
location  as  a  variable.  Of  the  fifty-six  ANOVA's  completed, 
nineteen  (19)  or  34%  indicated  a  significant  difference  in 
recommendations  for  instructional  media  competencies  among 
all  secondary  education  teachers  in  the  states  of  Hawaii,  Oregon 
and  Utah  at  the  .05  level  of  confidence  or  above.  Thirty-seven 
(37)  or  66%  indicated  no  significant  difference.  A  Duncan 
Multiple  Range  Test  was  conducted  for  each  of  the 
competencies  indicating  significant  difference. 

(Tables  are  available  upon  request.) 

3.  Is  there  a  difference  in  the  perceived  value  of 
instructional  media  use  in  the  classroom  among 
teachers  in  secondary  education  teaching  disciplines 
based  on  the  state  in  which  they  are  teaching? 

A  one-way  analysis  of  variance  was  computed  for  each 
teaching  location  namely  Hawaii,  Oregon  and  Utah.  For  the 
states  of  Hawaii  and  Oregon  there  were  no  significant 
differences  in  the  perceived  value  of  instructional  media  use  in 
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the  classroom  found  among  teachers  in  the  secondary  education 
teaching  disciplines. 

For  the  state  of  Utah,  a  significant  difference  in  the 
perceived  value  of  instructional  media  use  in  the  classroom 
among  teachers  in  the  secondary  education  teaching  disciplines 
was  found  at  the  .0008  level  of  confidence.  A  Duncan  Multiple 
Range  Test  was  computed  to  determine  which  teaching 
disciplines  were  significantly  different.  The  result  of  the  Duncan 
Procedure  indicates  that  Science  teachers  in  the  state  of  Utah 
with  a  mean  =  4.40  are  significantly  different  than  Music 
teachers  with  a  mean  =  2.25,  Physical  Education  teachers  with  a 
mean  =  3.14,  Business  teachers  with  a  mean  =  3.39  and 
Math/Computer  Science  teachers  with  a  mean  =  3.44.  In 
addition,  all  other  teachers,  ie.,  Health,  Home  Economics, 
Industrial  Arts.  Art,  Language  Arts,  Foreign  Language,  Social 
Science  and  Math/Computer  Science  are  significantly  different 
than  Music  teachers. 

(Tables  are  available  upon  request.) 

4.  Is  there  any  difference  in  instructional  media 

competency  recommendations  based  on  perceived 
value  of  instructional  media  use  in  the  classroom? 

The  data  relative  to  perceived  value  of  instructional  media 
use  in  the  classroom  was  recoded  from  continuous  to 
categorical.  Responses  of  0,  1,  2,  and  3  were  grouped  together 
into- category  l=low  perceived  value  of  instructional  media  use  in 
the  classroom.  Response  4  became  category  2=medium 
perceived  value  of  instructional  media  use  in  the  classroom. 
Response  5  became  category  3=high  perceived  value  of 
instructional  media  use  in  the  classroom.  There  were  one- 
hundred  and  fifty-one  (151)  respondents  or  36%  in  category  1- 
low  perceived  value;  one-hundred  and  forty  (140)  respondents 
or  34%  in  category  2-medlum  perceived  value;  and  one-hundred 
and  twenty-five  (125)  respondents  or  30%  in  category  3-high 
perceived  value  of  instructional  media  use  in  the  classroom. 

After  recoding,  a  one-way  analysis  of  variance  was 
computed  for  each  of  the  fifty-bix  (56)  instructional  media 
competencies.  Of  the  fifty-six  ANOVA's  completed,  forty-one 
(41)  or  73%  indicated  a  significant  difference  in 
recommendations  of  instructional  media  competencies  among 
teachers  having  low,  medium  and  high  perceived  value  of 
instructional  media  use  in  the  classroom  at  the  .05  level  of 
confidence  or  above. 

(Tables  are  available  upon  request.) 
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5.  Is  there  any  difference  in  instructional  media 
competency  recommendations  by  teachers  in 
secondary  education  teaching  disciplines  based  on 
perceived  value  of  instructional  media  use  in  the 
classroom? 

A  one-way  analysis  of  variance  was  computed  for  each  of 
the  fiity-six  instructional  media  competencies  controlling  for 
each  of  three  categories  (low,  medium  and  high)  of  perceived 
value  of  instructional  media  in  the  classroom.  A  total  of  one- 
hundred  and  sixty-eight  (168)  ANOVA's  were  completed. 

Selecting  only  teachers  with  a  low  perceived  value  of 
instructional  media  use  in  the  classroom  twenty-two  (22)  or 
39%  of  the  recommendations  for  instructional  media 
competencies  tested  significantly  different  among  secondary 
education  teaching  disciplines  at  the  .05  level  of  confidence  or 
higher.  Thirty-four  (34)  or  6 1%  of  the  instructional  media 
competencies  showed  no  significant  difference  among  teaching 
disciplines. 

Selecting  only  teachers  with  a  medium  perceived  value  of 
instructional  media  use  in  the  classroom  sixteen  (16)  or  29%  of 
the  recommendations  for  instructional  media  competencies 
tested  significantly  different  among  secondary  education 
teaching  disciplines  at  the  .05  level  of  confidence  or  higher. 
Forty  (40)  or  71%  of  the  instructional  media  competencies 
showed  no  significant  difference  among  teaching  disciplines. 

Selecting  only  teachers  with  a  high  perceived  value  of 
instructional  media  use  in  the  classroom  three  (3)  or  5%  of  the 
recommendations  for  instructional  media  competencies  tested 
significantly  different  among  secondary  education  teaching 
disciplines  at  the  .05  level  of  confidence  or  higher.  Fifty- three 
(53)  or  95%  of  the  instructional  media  competencies  showed  no 
significant  difference  among  teaching  disciplines. 

(Tables  are  available  upon  request.) 

To  assist  with  qualitative  analysis  and  determination  of 
appropriate  lists  of  instructional  media  competencies  by 
teaching  disciplines,  a  series  of  means  tables  (available  upon 
request)  were  developed  of  all  fifty-six  instructional  media 
competencies  by  teaching  disciplines. 

Table  4  presents  the  breakdown  of  the  recommendation 
scale  by  percentiles.  This  provides  some  criteria  for  the 
selection  of  instructional  media  competencies  either  for  the 
total  population  of  secondary  education  teaching  disciplines  or 
for  individual  teaching  disciplines.  A  mean  of  4.00  or  better  is  at 
or  above  the  80th  percentile  and  represents  a  very  strong 
recommendation.  A  mean  between  3.50  and  3.95  or  the  70th  to 
79th  percentile  represents  a  strong  recommendation  for  any 
given  instructional  media  competency.  Table  5  presents  a 
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summary  of  the  competencies  by  teaching  discipline  and  the 
selection  of  each  competency  by  discipline  at  or  above  the  70th 
and  80th  percentiles.  The  competencies  selected  for  the  total 
population  are  from  those  indicating  no  significant  difference 
from  the  analysis  of  variance  described  above. 


CONCLUSIONS  AND  RECOMMENDATIONS 

The  analysis  of  the  data  collected  presents  evidence  to 
support  several  conclusions  regarding  teaching  disciplines, 
instructional  media  competencies  and  perceived  value  of  media 
use  in  the  classroom.  The  first  hypothesis  focused  on  the 
significant  differences  among  the  teachers  of  secondary 
education  teaching  disciplines  and  their  recommendations  of 
instructional  media  competencies  to  be  taught  to  pre-service 
teacher  education  students  in  their  disciplines.  While  the  data 
does  not  indicate  differences  for  the  total  set  of  media 
competencies  there  is  enough  evidence  to  make,  a  general 
conclusion. 

Conclusion  1 

The  teaching  discipline  influences 
recommendations  by  inservice  teachers  of 
secondary  education  for  instructional  media 
competencies  to  be  included  in  a  pre-service 
teacher  education  program. 

Research  question  one,  ("Is  there  a  difference  in 
instructional  media  competency  recommendations  bv  teachers 
in  secondary  education  teaching  disciplines  based  on  the  state 
in  which  they  are  teaching?"),  and  two,  ("Is  there  any  difference 
in  instructional  media  competency  recommendations  among  all 
teachers  in  Hawaii  Oregon  or  Utah?"),  are  concerned  with  the 
teaching  location  namely  Hawaii,  Oregon  and  Utah  and  the 
recommendations  of  instructional  media  competencies.  The 
data  indicates  a  significant  difference  exists  among  teachers  of 
secondary  education  teaching  disciplines  in  the  states  of  Hawaii, 
Oregon  and  Utah  regarding  their  recommendations  of 
instructional  nedia  competencies  to  be  included  in  a  pre- 
service  teacher  education  program.  For  Hawaii  and  Utah  there 
were  eleven  (11)  competencies  indicating  significant  difference, 
while  in  Oregon  there  were  twenty-two  (22).  Careful  review  of 
the  data  indicates  no  evidence  to  suggest  any  similarities  among 
teaching  disciplines  from  the  different  states.  The  data 
presented  regarding  recommendations  of  instructional  media 
competencies  by  all  teachers  in  Hawaii,  Oregon  and  Utah 
indicates  that  of  the  nineteen  (19)  media  competencies 
indicating  a  significant  difference,  teachers  from  Hawaii  have 
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the  highest  recommendation  means  for  ail  nineteen  (19)  with 
Oregon  teachers  having  the  lowest  recommendation  means  for 
eighteen  (18)  of  the  nineteen  (19)  competencies  presented. 

The  second  hypothesis  focused  on  the  significant 
differences  among  teachers  of  secondary  education  teaching 
disciplines  and  their  perceived  value  of  instructional  media  use 
in  the  classroom.  The  data  indicates  a  significant  difference 
among  teachers  of  secondary  education  teaching  disciplines 
regarding  their  perceived  value  of  instructional  media  use  in  the 
classroom.  Table  3  provides  a  ranking  of  teaching  disciplines 
from  high  to  low  of  perceived  value  of  instructional  media  use  in 
the  classroom.  The  hypothesis  was  rejected  at  the  .0002  level  of 
confidence  which  gives  strong  evidence  for  a  second  conclusion 
from  this  study. 

Conclusion  2 

The  teaching  discipline  influences  the 
perceived  value  of  instructional  media  use  in  the 
classroom  of  inservice  teachers  of  secondary 
education. 

As  was  the  case  for  research  questions  one  and  two  as 
described  above,  research  question  three,  ("Is  there  a  difference 
in  the  perceived  value  of  instructional  media  use  in  the 
classroom  among  teachers  in  secondary  education  teaching 
disciplines  based  on  the  state  in  which  thev  are  teaching?") . 
focused  on  the  consideration  of  teaching  location.  The  findings 
indicate  that  there  were  no  differences  among  teachers  in 
Hawaii  or  Oregon  regarding  perceived  value  of  instructional 
media  use  in  the  classroom.  However,  there  is  a  significant 
difference  in  the  perceived  value  of  instructional  media  use  in 
the  classroom  among  teachers  of  secondary  education  teaching 
disciplines  in  Utah  at  the  .01  level  of  confidence.  Music 
teachers,  with  a  value  mean  of  2.25,  have  the  lowest  perceived 
value  of  instructional  media  use  in  the  classroom  while  science 
teachers,  with  a  value  mean  of  4.40,  have  the  highest  perceived 
value.  No  conclusions  are  being  suggested,  only  a  recognition 
that  differences  exist  among  teachers  in  Utah  that  were  not 
evidenced  in  either  Hawaii  or  Oregon. 

Additional  information  was  developed  relative  to  the 
consideration  of  perceived  value  of  instructional  media  use  in 
the  classroom  by  research  question  four;  wIg  there  any 
difference  in  instructional  media  competency  recommendations 
based  on  perceived  value  of  instructional  media  use  in  the 
classroom."  By  recoding  the  perceived  value  of  instructional 
media  data  into  categorical  levels  of  low,  medium  and  high  it 
was  possible  to  further  analyze  the  teachers  recommendations  of 
instructional  media  competencies  to  be  included  in  a  pre- 
sence teacher  education  program.  The  findings  indicate  that 
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there  is  3ignificant  difference  among  teachers  having  low, 
medium  and  high  perceived  value  of  instructional  media  use  in 
the  classroom  regarding  recommendations  of  forty-one  (41)  of 
the  fifty-six  (56)  instructional  media  competencies  studied.  ^In 
general,  all  recommendations  from  teachers  having  a  "high" 
perceived  value  of  instructional  media  use  in  the  classroom  had 
a  higher  mean  than  either  of  the  other  two  categories.  These 
data  provides  evidence  to  support  the  following  conclusion. 

Conclusion  3 

The  perceived  value  of  instructional  media 
v.oe  in  the  classroom  by  secondary  education 
teachers  influences  their  recommendations  of 
instructional  media  competencies  to  be  included 
in  a  pre-service  teacher  education  program. 

The  data  collected  and  analyzed  regarding  research 
question  five,  Hg  there  any  difference  in  instructional  mgcjigk 
competency  recommendations  by  teachers  In  secondary 
education  teaching  disciplines  based  on  perceived  value  Qi 
instructional  media  use  in  the  classroom?"),  also  support 
conclusion  3  regarding  the  influence  of  perceived  value  of 
instructional  media  use  in  the  classroom  on  media  competency 
recommendations.  There  were  twenty- two  (22)  media 
competencies  which  indicate  significant  differences  among 
teachers  having  a  low  perceived  value  of  instructional  media  use 
in  the  classroom.  For  teachers  with  a  medium  perceived  value 
of  instructional  media  use  there  were  sixteen  (16)  media 
competencies  indicating  a  significant  difference.  There  were 
only  three  media  competencies  indicating  significant 
differences  among  secondary  education  teachers  having  a  high 
perceived  value.  This  set  of  data  suggests  that  the  higher  the 
perceived  value  of  Instructional  media  use  in  the  classroom  the 
fewer  differences  there  are  among  teachers  of  secondary 
education  teaching  disciplines  for  instructional  media 
competencies  to  be  included  in  a  pre-service  teacher  education 
program. 

The  intent  of  this  study  was  to  determine  the  instructional 
media  competencies  that  inservice  teachers  of  secondary 
education  teaching  disciplines  recommend  for  pre-service 
teachers  in  their  discipline.  The  primary  purpose  was  to 
identify  the  instructional  media  competencies  common  to  all 
teaching  disciplines  and  in  addition,  the  media  competencies 
unique  to  each  discipline  individually.  The  final  conclusion  is  in 
response  to  the  original  objectives  of  the  study.  Table  5 
presents  a  summary  of  the  instructional  media  competencies 
that  are  recommended  by  all  teachers  of  secondary  education 
teaching  disciplines  and  those  unique  to  specific  teaching 
disciplines.  These  are  presented  in  support  of  the  following 
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general  conclusion  about  instructional  media  competencies 
significant  to  teachers  of  secondary  education  teaching 
disciplines. 

Conclusion  4 

There  are  instructional  media  competencies 
that  are  common  to  all  secondary  education 
teaching  disciplines  as  well  as  instructional 
media  competencies  that  are  unique  to  each  of 
twelve  secondary  education  teaching  disciplines. 


Recommendations 

The  recommendations  set  forth  in  this  study  are  based  on 
two  assumptions.  According  to  the  literature  reviewed, 
instructional  media,  if  properly  used  can  enhance  the 
teaching/learning  process  by  increasing  the  amount  of  learning 
or  reducing  the  amount  of  time  necessary  to  accomplish  the 
desired  outcome.  Second,  inservice  teachers  of  secondary 
education  teaching  disciplines  have  knowledge  and  experience 
that  can  provide  a  better  understanding  of  the  needs  of  pre- 
service  teachers  as  they  are  preparing  to  enter  the  schools,  as 
indicated  by  research  reviewed  and  the  findings  of  this  study. 
Given  these  assumptions,  the  following  recommendations  are 
derived  from  the  results  of  this  study. 

Recommendation  1 

Instructors  of  secondary  education  teaching 
discipline  methods  courses  and  instructors  of 
instructional  media  in  institutions  of  higher 
education  offering  teacher  education  programs 
should  jointly  design  and  develop  learning  activities 
that  will  provide  pre-service  teacher  education 
students  in  specific  teaching  disciplines  the 
instructional  media  competencies  identified  as 
necessary  for  their  discipline. 

The  recommendation  described  above  could  be 
accomplished  by  carefully  selecting  those  instructional  media 
competencies  that  are  best  developed  in  a  formal  instructional 
media  course  and  those  that  could  best  be  presented  in  the 
methods  courses.  Each  should  reinforce  and  support  the  other. 
It  is  important  to  recognize  that  one  of  the  significant 
components  of  these  courses  and  activities  must  be  effective  and 
appropriate  modeling  of  these  skills  and  competencies  by  the 
instructors  involved.  Learning  about  something  is  not  the  same 
as  having  a  continuous  example  of  its  application  in  a  realistic 
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setting  that  represents  the  environment  in  which  pre-service 
teacher  education  students  will  be  working. 

Recommendation  2 

Specific  learning  activities  should  be  developed 
that  will  enhance  the  pre-service  teacher  education 
students  perception  of  the  value  of  the  use  of 
instructional  media  in  the  classroom. 

This  recommendation  could  be  accomplished  by 
instructional  media  instructors  and  methods  instructors 
providing  their  students  opportunities  to  be  directly  involved  in 
educational  innovation  projects.  These  could  focus  on  the 
application  of  computer  or  video  technology  to  the 
teaching/learning  process  or  in  the  design  and  delivery  of 
instruction  via  telecommunication  technology.  The  benefits  of 
this  type  activity  would  be  two  fold.  First,  the  students  would 
have  opportunity  to  see  technology  being  applied  to  the 
teaching/learning  process  in  a  "real"  setting.  Second,  by  being 
involved  on  a  participatory/ contributive  basis,  the  students  will 
develop  a  better  understanding  of  the  benefits  of  such  programs 
in  terms  of  direct  learner  improvement.  By  being  involved  they 
can  develop,  to  a  degree,  a  sense  of  ownership  in  the  outcomes 
and  ideas  of  the  project.  It  should  be  pointed  out  that  such 
projects  do  not  necessarily  have  to  be  high  technology  based. 
Providing  pre-service  teacher  education  students  opportunity  to 
develop  and  apply  new  applications  of  older  and  more  simple 
technology  in  the  delivery  of  instructional  activities  can  be  just 
as  rewarding. 

Recommendation  3 

State  Teaching  Certification  Requirements 
should  be  written  to  require  evidence  that  secondary 
education  teachers  have  both  general  instructional 
media  competencies  as  well  as  specific  skills  that 
are  significant  to  their  teaching  discipline. 

The  implementation  of  this  recommendation  would 
encourage  schools  and  colleges  of  education  to  provide 
programs  that  offer  courses  in  instructional  media  as  well  as 
methods  courses  for  each  teaching  discipline.  It  would  also 
encourage  cross-departmental  cooperation  in  the  development 
and  offering  of  learning  activities  designed  to  provide  pre- 
service  teacher  education  students  the  skills  and  knowledge 
necessary  to  meet  the  challenges  of  todays  teaching  profession. 
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A  Personal  Note 


Considering  the  complexity  of  the  teaching/learning 
process,  it  would  seem  that  serious  energy  and  thought  should 
be  expended  on  behalf  of  reform  of  pre-service  teacher 
education  programs.  Teachers  today  are  facing  the  challenge  of 
preparing  their  students  for  a  world  that  none  of  them  can  even 
begin  to  envision.  This  requires  a  level  of  preparation  and  skill 
that  has  not  been  demanded  in  the  past.  In  order  for  teacher 
education  programs  to  meet  their  responsibility  to  prepare 
future  teachers,  it  is  necessary  to  develop  a  greater  sense  of  and 
commitment  to,  cooperative  efforts  among  departments  and 
faculty.  Effort  needs  to  be  made  that  will  breakdown  the 
departmentalization  and  compartmentalizing  of  the  various 
components  within  the  university  that  make  up  a  typical  teacher 
education  program.  Pre-service  teacher  education  students 
need  to  experience  an  integrated,  cooperative  program  where 
each  contributor  is  seen  as  professional  and  integral  to  the  total 
educational  system.  Subject  matter  specialists,  methods 
instructors,  classroom  management  specialists,  media 
specialists  and  the  other  applications  specialists  must  work 
together  to  provide  the  student  with  an  understanding  of  how 
each  adds  to  the  success  of  system. 

A  final  observation  from  this  research  would  be  that 
inservice  teachers  place  value  on  the  skills  and  instructional 
media  competencies  that  they  use  in  their  classrooms.  They 
also  seem  to  be  saying  that  one  of  the  ways  they  developed  a 
sense  of  value  for  their  use  was  through  example  (or  non- 
example)  presented  by  their  methods  instructors.  The  methods 
teacher,  working  in  cooperation  with  the  instructional 
media/ technology  information  specialist  has  a  unique 
opportunity  to  present  methods,  instructional  approaches  and 
media  utilization  in  a  way  that  can  not  be  duplicated  elsewhere. 
When  the  pre-service  teacher  of  a  particular  discipline  can  see 
methods  and  materials  being  used  in  the  context  of  the  subject 
of  interest,  they  are  more  likely  to  attempt  to  utilize  or  replicate 
that  in  their  own  classroom  when  the  time  comes. 

The  preparation  of  tomorrows  teachers  is  a  serious 
challenge.  With,  the  application  of  effective  research,  technology 
and  an  ever  Increasing  understanding  of  the  process  of  learning 
the  task  will  be  accomplished.  Teacher  education  has  never 
before  been  prewSented  with  such  an  opportunity  to  make  a  life 
altering  contribution  to  the  students  of  tomorrow.  Pre-service 
teachers  today  must  be  given  the  tools  and  knowledges 
necessary  to  provide  learning  experiences  for  the  students  of 
tomorrow.  Those  students  will  be  required  to  do  more  than 
rote  recitation;  they  will  need  to  function  in  a 
technological/information  based  society  that  demands  high  level 
thinking  skills.  The  traditional  textbook  and  lecture  bound 
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teacher  preparation  program  does  not  provide  the  level  of  skills 
and  knowledge  required.  A  change  is  required  for  the  teacher 
education  program  of  tomorrow.  Cooperation  and  integration 
among  all  faculty  within  the  teacher  education  program  will  lead 
to  a  far  better  prepared  teacher  of  tomorrow. 
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Table  i:  Fifty-six  Instructional  Media  Competencies 


I.  Principles  of  Communication, 
Selection,  Evaluation  and  Research 

1 .  Communication  Theory 

2.  Design  and  Layout  of  Visual 
Materials 

3.  Instructional  Design  Theory 
and  Practice 

4.  Media  Selection  and  Evaluation 
Criteria 

5.  Impact  of  Technology  on  Education 

6.  Implications  of  Instructional  Media 
Research 

7.  Future  Trends  of  Media  and 
Technology  In  Education 

8.  Copyright  Laws  and  Education 

II.  How  to  PRODUCE  Instructional 
Media  Materials 

9.  Lettering  for  Instructional  Materials 

10.  Mounting  Visuals 

1 1 .  Laminating  Visuals 

12.  Machine  Produced  Overhead 
Transparencies 

13.  Handmade  Overhead 
Transparencies 

14.  Display  Boards  (Bulletin  Boards. 
Displays,  etc.) 

15.  Duplicating  Inst.  Materials 
(Dittos.  Xerox,  etc.) 

16.  Illustration  and  Enlargement 
Techniques 

17.  Manlpulatives  (Mathematlc 
materials,  etc.) 

18.  Audio  Recording 

19.  Video  Recording  (off-air  recording) 

20.  Video  Programming 
(Producing  own  programs) 

21.  Still  photography 

22.  Slide/ tape  programs 

23.  Computer  Assisted  Instruction 

24.  Computer  Programming 

25.  Computer  Graphics 

26.  Games,  simulations  and  media  kits 


HI.  How  to  UTILIZE  Instructional  Media 
Materials 

27.  Non-projected  visuals 

28.  Overhead  Transparencies 

29.  Display  Boards  (Bulletin  Boards,  etc.) 

30.  Flip  Charts 

31.  Chalkboards 

32.  Duplicated  Materials 
(Dittos,  Xerox,  etc.) 

33.  Manlpulatives 
(Mathematlc  materials,  etc.) 

34.  Audio  Recordings 

35.  Instructional  Films  and  Videos 
(tape  &  disc) 

36.  Broadcast  Television 

37.  Slides 

38.  Filmstrips 

39.  Computer  Assisted  Instruction 

40.  Computer  Interactive  Video  Programs 

41.  Games  and  Simulations 

42.  Free  and  Inexpensive  Materials 

43.  Field  Trips  and  Community  Resources 

IV.  How  to  OPERATE  Instructional 
Media  Equipment 

44.  Overhead  Projectors 

45.  Spirit  Duplicators  (Ditto) 

46.  Opaque  Projectors 

47.  Cassette  Tape  Recorders 

48.  Record  Players 

49.  Video  Tape  Recorders 

50.  Video  Camcorder  Systems 

51.  Video  Editing  Systems 

52.  16mm  Motion  Picture  Projectors 

53.  2X2  Slide  Projectors 

54.  Filmstrip  Projectors 

55.  Computer  Interactive  Video  Systems 

56.  Microcomputer  Overhead 
Projector  LCD  Systems 
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Table  2:  Response  Frequencies 


HAWAII 

OREGON 

UTAH 

TOTAL 

Alt 

Q 

Q 
O 

1  0 

Business 

1 1 

9 

13 

33 

Foreign  Language 

17 

8 

14 

39 

Health 

15 

1 1 

12 

38 

Home  Economics 

Q 

1  7 

41 

Industrial  Arts 

9 

7 

24 

Language  Arts 

15 

7 

14 

36 

Math /Computer  Science 

26 

16 

16 

58 

Music 

9 

11 

8 

28 

Physical  Education 

12 

11 

7 

30 

Science 

12 

16 

10 

38 

Social  Science 

5 

8 

22 

Total 

153 

126 

137 

416 

0 
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Table  3      Relative  Ranking  of  Teaching  Disciplines 
by  Perceived  Value  of  Media  Use  in  the 
Classroom 


m 


w 


m 


IWIW!!!.' !!!!!!!!  •■p 
1 Mft 


T 


T 


,..„  L—  1  1  I 


I      I      I  I 

1        2  3  


4.1667  Industrial  Arts 
4.1220  Home  Economics 
4.0526  Health 
4.0000  Social  Science 
3.9737  Science 
3.8718  Foreign  Language 
3.8611  Language  Arts 
3.6667  Business 
3.6552  Art 
3.5667  P.E. 

3.5000  Math/Comp.  Sc. 
2.8928  Music 
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Table  4  Breakdown  of  Recommendation 
Scale  by  Percentile   


to 

Qh 


95th- 

4.75 

90th- 

4.50 

85th- 

4.25 

80th- 

z  A.QQ 

75th- 

—  3.75 

70th- 

z  am 

65th- 

3.25 

60th- 

3.00 

55th- 

2.75 

50th- 

—  2.50 

45th- 

2.25 

40th- 

2.00 

35th- 

1.75 

30th- 

1.50 

25th- 

-  1.25 

20th- 

1.00 

15th- 

0.75 

10th- 

0.50 

5th- 

0.25 

CJ 

CO 
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Table  5    Summary:  Selection  of  Instructional  Media  Competencies 


**  =  At  or  above  80th  percentile. 

*  =  Between  70th  and  79th  percentile. 
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o 

os 

l! 
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8 
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3) 


j  is 

g  i  s 

03  o  T 
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3 


II 

a  w 


Foundiitlon  and  Thconr  Competencies: 


1,  Communication  Theory 


2,  Design  and  Layout  of  Visual  Materials 


3.  Instructional  Design  Theory  and  Practice 


4.  Media  Selection  and  Evaluation  Criteria 


5 .  Impact  of  Technology  on  Education 


6.  Implications  of  Inst,  Media  Research 


7.  Future  Trends  of  Media  and  Tech,  in  Educ 


8.  Copyright  Laws  and  Education 


a 


2 


8 


9.  Lettering  for  Instructional  Materials 

10.  Mounting  Visuals 

* 

1 1 .  Laminating  Visuals 

* 

* 

* 

12.  Machine  Prod.  Transparencies 

* 

** 

* 

1 3 .  Handmade  Transparencies 

* 

* 

* 

14.  Disnlav  Boards  (B.FL  etc  ^ 

** 

* 

** 

15.  Duplicating  Inst.  Materials  (Dittos,  Xerox,  etc.) 

* 

♦* 

♦ 

* 

* 

* 

** 

* 

** 

16.  Illustration  and  Enlargement  Technioucs 

* 

*♦ 

* 

* 

17.  Manipulative*  (Mathematics  materials,  etc.) 

** 

* 

18.  Audio  Recording 

** 

1 9.  Video  Recording  (off-air  recording) 

* 

** 

** 

* 

* 

20.  Video  Programming  (Producing  own  programs) 

♦* 

* 

* 

** 
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Table  5    Summary:  Selection  of  Instructional  Media  Competencies  (cont.) 


**  =  At  or  above  80th  percentile. 

*  =  Between  70th  and  79th  nercenLlle 

Total 

Population 
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Business 

Foreign 
Language 
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Industrial 
Arts 
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Computer 

Music 
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Education 
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Table  5    Summary:  Selection  of  Instructional  Media  Competencies  (cont.) 


44.  Overhead  Pmjprtnr* 

jtf^Snirii  Diinlinatnrs  (V>\un\ 

Opaque  Bcojecmcs  

47.  Cassette  Tape  Rfttnrrim 

Aft   Rprnrd  Playprv 

49.  Video  Tape  Remnlnrs 

♦  ♦♦ 

*♦ 

SO    Vi/Wk  PammrHpr  Sy&l£flU   

♦  ♦♦ 

.51.  Video  Editing  Systems 

* 
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*♦ 
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MEDIA  COMPETENCY  RECOMMENDATION  SURVEY 


INSTRUCTIONS 

The  puipose  of  this  survey  is  to  determine  instructional  media  competencies  to  be  included  in  a 
pre-scrvice  teacher  education  program  as  recommended  by  inservice  secondary  teachers.  Please 
respond  to  items  as  indicated,  using  the  scale  described  for  each  category. 

Your  responses  should  be  based  on  your  experience  as  a  teacher  in  your  current  teaching 
discipline.  It  would  be  helpful  if  you  thougtu  in  terms  of  how  you  would  structure  an  introductory 
instructional  media  course  for  pre-service  teachers  preparing  to  teach  in  your  teaching  discipline. 
If  you  are  teaching  in  more  than  one  discipline  please  respond  based  on  your  university  major. 


LOCATION    (Check  one) 


HAWAII  D 
OREGON  Q 
UTAH  D 


SCHOOL  TYPE:  (Check  one) 


HIGH  SCHOOL  (9-12)  [] 
JR.  HIGH  SCHOOL  (7-8)  □ 
COMBINED  7-12  Q 


YEARS  TEACHING 


NUMBER  OF  YEARS 
TEACHING  FULL 
TIME? 


SCHOOL  ENROLLMENT 


MEDIA  COURSES 


APPROXIMATE 

SCHOOL 

ENROLLMENT? 


LZZI 


NUMBER  OF  CREDIT  HOURS  COMPLETED  IN 
INSTRUCTIONAL  MEDIA?  j  j  Sat^ 


TEACHING  DISCIPLINE:  (Check  the  one  that  best  describes  your  assignment.) 


ART 

BUSINESS 

COMPUTER  SCIENCE 
FOREIGN  LANGUAGE 
HEALTH 

HOME  ECONOMICS 
INDUSTRIAL  ARTS 


□ 
□ 
□ 
□ 
□ 
□ 
□ 


LANGUAGE  ARTS 

MATHEMATICS 

MUSIC 

PHYSICAL  EDUCATION 
SCIENCE 
SOCIAL  SCIENCE 
OTHER  


Are  you  the  building  media  specialist? 
yesp        no  CD 


□ 
□ 
□ 
□ 
□ 
□ 
□ 


Value  of  Media 


In  General  How  Would  You  Rank 

the  Valueof  Instructional  Media/Technology 

in  your  Classroom?  


Not 
Valued 
0 


Highly 
Valued 
4  5 


32 


4  - 


The  following  instructional  media  competencies  are 
found  in  most  introductory  media  courses.  Using  the 
scale  to  the  right,  please  indicate  your  recommendation 
for  their  continued  inclusion  in  a  pre-service  teacher 
education  program  based  on  your  experience  in  your 
particular  teaching  discipline. 


Not 

Highly 

Recommended 

Recommended 

0  1 

2     3     4  5 

I.  Principles  of  Communication,  Selection,  Evaluation  and  Research 


Not 

Highly 

Recommended 

Recommended 

1 .   Communication  Theory 

0  1 

2 

3 

4 

5 

2.   Design  and  Layout  of  Visual  Materials 

0  1 

1 

Li 

3 

4 

5 

3.   Instructional  Design  Theory  and  Practice 

0  1 

2 

3 

4 

5 

4.   Media  Selection  and  Evaluation  Criteria 

0  1 

2 

3 

4 

5 

5.   Impact  of  Technology  on  Education 

0  1 

2 

3 

4 

5 

6.   Implications  of  Instructional  Media  Research 

0  1 

2 

3 

4 

5 

7.   Future  Trends  o  f  Media  and  Technology  in  Education 

0  I 

2 

3 

4 

5 

8.   Copyright  Laws  and  Education 

0  1 

2 

3 

4 

5 

9.  Other 

0  1 

2 

3 

4 

5 

.  How  to  PRODUCE  Instructional  Media  Materials 

iNOl 

Highly 

Recommended 

Recommended 

10.  Lettering  for  Instructional  Materials 

0  1 

2 

3 

4 

5 

11.  Mounting  Visuals 

0  1 

2 

3 

4 

5 

12.  Laminating  Visuals 

0  1 

2 

3 

4 

5 

13.  Machine  Pruduced  Overhead  Transparencies 

0  1 

2 

3 

4 

5 

14.  Handmade  Overhead  Transparencies 

0  1 

2 

3 

4 

5 

li.  Display  Boards  (Bulletin  Boards,  Displays,  etc.) 

0  1 

2 

3 

4 

5 

16.  Duplicating  Inst.  Materials  (Dittos,  Xerox,  etc.) 

0  1 

2 

3 

4 

5 

17.  Illustration  and  Enlargement  Techniques 

0  1 

2 

3 

4 

5 

18.  Manipulatives  (Mathematic  materials,  etc.) 

0  1 

2 

3 

4 

5 

19.  Audio  Recording 

0  1 

2 

3 

4 

5 

2C.  Video  Recording  (off-air  recording) 

0  1 

2 

3 

4 

5 

21.  Video  Programming  (Producing  own  programs) 

0  1 

2 

3 

4 

5 

22.  Still  photography 

0  1 

2 

3 

4 

5 

23.  Slide/tape  programs 

0  1 

2 

3 

4 

5 

24.  Computer  Assisted  Instruction 

0  1 

2 

3 

4 

5 

25.  Computer  Programming 

0  1 

2 

3 

4 

5 

26.  Computer  Graphics 

0  1 

2 

3 

4 

5 

27.  Games,  simulations  and  media  kits 

0  1 

2 

3 

4 

5 

Other 

0  1 

1  2 

3 

4 

5 

9 
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IIL  How  to  UTILIZE  Instructional  Media  Materials  (apply  media  in  the  teaching/learning  process) 


Not 

Highly 

Recommended 

Recormrcnded 

29.  Non-projected  visuals 

0  1 

2 

3 

4 

5 

30.  Overhead  Transparencies 

0  1 

2 

3 

4 

5 

31.  Display  Boards  (Bulletin  Boards,  etc.) 

0  1 

2 

3 

4 

5 

32.  Flip  Charts 

0  1 

2 

3 

4 

5 

33.  Chalkboards 

0  1 

2 

3 

4 

<J 

34.  Duplicated  Materials  (Dittos,  Xerox,  etc.) 

0  1 

2 

3 

4 

5 

35.  Manipulates  (Mathematic  materials,  etc.) 

0  1 

2 

3 

4 

5 

36.  Audio  Recordings 

0  1 

2 

3 

4 

5 

37.  Instructional  Films  and  Videos  (tape  &  disc) 

0  1 

2 

3 

4 

5 

38.  Broadcast  Television 

0  1 

2 

4 

39.  Slides 

0  1 

2 

3 

4 

5 

40.  Filmstrips 

0  1 

2 

3 

4 

5 

41.  Computer  Assisted  Instruction 

0  1 

2 

3 

4 

5 

42.  Computer  Interactive  Video  Programs 

0  1 

2 

3 

4 

5 

43.  Games  and  Simulations 

0  1 

2 

3 

4 

5 

44.  Free  and  Inexpensive  Materials 

0  1 

2 

3 

4 

5 

45.  Field  rrips  and  Community  Resources 

0  1 

2 

3 

4 

5 

46.  Other 

0  1 

2 

3 

4 

5 

/.  How  to  OPERATE  Instructional  Media  Equipment 

Not 

Highly 

Recommended 

Recommended 

47.  Overhead  Projectors 

0  1 

2 

3 

4 

5 

48.  Spint  Duplicators  (Ditto) 

0  1 

2 

3 

4 

5 

49.  Opaque  Projectors 

0  1 

2 

3 

4 

5 

50.  Cassette  Tape  Recorders 

0  1 

2 

3 

4 

5 

51.  Record  Players 

0  1 

2 

3 

4 

5 

52.  Video  Tape  Recorders 

0  1 

2 

3 

4 

5 

53.  Video  Camcorder  Systems 

0  1 

2 

3 

4 

5 

54.  Video  Editing  Systems 

0  1 

2 

3 

4 

5 

55.  16mm  Motion  Picture  Projectors 

0  1 

2 

? 

4 

5 

56.  2X2  Slide  Projectors 

0  1 

2 

3 

4 

5 

57,  Filmstrip  Projectors 

0  1 

2 

3 

4 

5 

58.  Computer  Interactive  Video  Systems 

0  1 

2 

3 

4 

5 

59.  Microcomputer  Ovethead  Projector  LCD  Systems 

0  1 

2 

3 

4 

5 

60.  Other 

0  1 

2 

3 

4 

5 

9 
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Instructional  Approach 


Please  rank  the  following  approaches  for  teaching  Instructional  media  competencies  In  the 
order  you  would  recommend  based  on  your  experience  in  your  particular  teaching  discipline. 

Ranking  (lst-Sth) 

1.  Formal  Courses  in  instructional  Media. 

2.  Media  Competencies  Integrated  within  the  Teaching  Methods  Courses. 

3.  Media  Competencies  Integrated  within  all  Education  Courses. 

4.  Combination  of  Formal  Courses  and  an  Integration  of  Media  Competencies 
within  the  Teaching  Methods  Courses. 

5.  Other  


Course  Emphasis 

The  following  categories  of  instructional  media  competencies  constitute  the  curriculum  in  an 
introductory  instructional  media  course.  Indicate  the  percentage  of  emphasis  that  you  would 
recommend  for  each  category,  (see  pages  2  and  3  for  reference) 


I.  Principles  of  Communication,  Selection, 
Evaluation  and  Research,  (sec  page  2) 

II.  How  to  Prod'^e  Instructional  Media 
Materials,  (seepage 2) 

IE.  How  to  Utilize  Instructional  Media 
Materials,  (see  page  3) 

IV.  How  to  Operate  Instructional  Media 
Equipment,  (see  page  3) 


<3o 


100% 


Non-Use  Factors   Check  all  the  factors  that  most  frequently  contribute  to  your 
descision  not  to  use  instructional  media  in  the  classroom. 


1. 

Textbook  materials  are  adequate. 

□ 

2. 

Do  not  believe  media  would  help. 

□ 

3. 

Media  is  too  time  consuming. 

□ 

4. 

Arranging  to  use  media  is  too  great  a  hassle. 

□ 

5. 

Media  hardware  are  too  difficult  to  operate. 

□ 

6. 

Media  materials  in  the  school  are  outdated. 

□ 

7. 

No  administrative  support  for  using  media. 

□ 

8. 

Other 

□ 

Thank  you  for  your  help.  Please  place  your  completed  questionnaire  in  the  self  addressed  prepaid  mailer  and  return  it  to  the 
researcher  at  your  earliest  convenience.  If  you  would  like  to  review  (he  results  of  the  study  please  include  your  name  and  address. 
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Learning  and  Semantic  Networks 

Our  semantic  networks  represent  our  knowledge  structures  which  enable  learners  to 
combine  ideas,  infer,  extrapolate  or  otherwise  reason  from  them.  Learning  consists  of  building 
new  structures  by  constructing  new  nodes  and  interrelating  them  with  existing  nodes  and  with 
each  other.  The  more  links  that  can  be  formed  between  existing  knowledge  and  new  knowledge, 
the  better  the  information  will  be  comprehended  and  the  easier  learning  will  be.  Learning, 
according  to  semantic  network  theory,  is  the  reorganization  of  the  learner's  knowledge  structure. 
During  the  process  of  learning,  the  learner's  knowledge  structure  begins  to  resemble  the  the 
knowledge  structures  of  the  instructors,  and  the  degree  of  similarity  is  a  good  predictor  of 
classroom  examination  performance  (Diekhoff,  1983;  Shavelson,  1974;  Thro,  1978).  Instruction 
then  may  be  conceived  of  as  the  mapping  of  subject  matter  knowledge  (usually  that  possessed  by  the 
teacher  or  expert)  onto  the  learner's  knowledge  structure. 

If  we  accept  the  conception  that  learning  is  involves  the  reorganization  of  the  learner's 
cognitive  structure,  then  instruction  involves  the  assimilation  of  the  expert's  knowledge  structure 
by  the  student.  In  order  to  help  students  reorganize  and  tune  their  knowledge  structures,  we  need 
instructional  strategies  for  depicting  and  displaying  appropriate  knowledge  structures  to  students 
and  tools  for  organizing  their  knowledge  structures.  Instructional  strategies  may  illustrate  or 
convey  appropriate  knowledge  structures  to  students,  whereas  the  tools  or  learning  strategies  may 
help  learners  to  acquire  and  refine  their  own  knowledge  structures  (Jonassen,  Beissner,  &  Yacci, 
in  press).  This  study  compares  the  effectiveness  of  an  instructional  strategy  which  displays 
appropriate  knowledge  structures  with  a  learning  strategy  that  engage  learners  in  defining 
knowledge  structures.  The  former  strategy  illustrates  key  concepts  and  their  interrelationships 
where  the  latter  requires  the  learners  to  identify  and  classify  relationships  between  key  concepts, 
which  plae  more  intelectual  responsibility  onto  the  students  for  meaning-making. 

The  learning  variable  that  is  being  investigated  here  is  structural  knowledge.  Structural 
knowledge  is  the  knowledge  of  how  concepts  within  a  domain  are  interrelated  (Diekhoff,  1983). 
Structural  knowledge  enables  learners  to  form  the  connections  t^at  they  need  to  describe  and  use 
scripts  or  complex  schemas.  It  is  a  form  of  conceptual  ^"owledL  .  that  mediates  the  translation  of 
declarative  knowledge  into  procedural  knowledge. 

Instructional  Strategies  vs.  Learning  Strategies 

An  instructional  strategy  is  a  method  or  technique  for  providing  instruction  to  learners. 
They  are  implemented  through  instructional  teactics  (Jonassen,  Grabinger,  &  Harris,  1991).  For 
instance,  an  instructional  strategy  may  recommend  motivating  the  learner  prior  to  instruction, 
which  may  call  for  tactics  such  as  arousing  learner  uncertainty,  asking  a  question  or  presenting 
a  picture.  A  strategy  aimed  at  teaching  a  concrete  concept  may  call  for  the  use  of  tactics  such  as 
matched  example-nonexample  pairs  or  deriving  the  criterion  attributes  from  a  set  of  examples. 
Instructional  strategies  provide  the  overall  plan  that  guides  the  selection  of  instructional  tactics 
that  facilitate  learning.  Instructional  strategies  may  be  divided  into  four  main  classes: 
contextualizing  instruction,  providing  learner  control,  organizing  and  cueing  learning,  and 
assessing  and  evaluating  learning.  Instructional  strategies  are  instructor-provided 
interventions  that  are  meant  to  stimulate  learner  processing  of  information. 
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Figure  1 


Learning  strategies  are  mental  operations  that  the  learner  may  use  to  acquire,  retain  and 
retrieve  different  kinds  of  knowledge  or  performance.  Figure  1  presents  a  taxonomy  of  learning 
strategies  that  may  be  used  by  learners  to  integrate  new  information  into  their  knowledge 
structures  and  restructure  and  synthesize  that  knowledge.  The  fundamental  difference  between 
instructional  strategies  and  learning  strategies  is  that  the  former  are  largely  mathemagenic  and 
the  latter  are  generative  (Jonassen,  1985).  Mathemagenic  instructional  strategies  control  the 
processing  of  learners  while  leading  them  to  learning,  so  they  result  in  anticipated  learning 
outcomes.  They  facilitate  acquisition  of  specific  content  knowledge  (intentional  learning)  but 
generally  do  not  affect  or  even  impede  the  acquisition  of  any  other  knowledge  (incidental 
learning). 

Learning  strategies,  on  the  other  hand,  are  generative,  that  is,  they  enable  learners  to  take 
an  active,  constructive  role  in  generating  meaning  for  information  by  accessing  and  applying 
prior  knowledge  to  new  material.  Wittrock's  (1978)  generative  hypothesis  asserts  that  meaning 
for  material  presented  by  computer  or  any  other  medium  is  generated  by  the  learner's  activating 
and  altering  existing  knowledge  structures  in  order  to  interpret  what  is  presented.  Cognitive 
learning  strategies  are  intended  to  increase  the  number  of  links  between  presented  information 
and  existing  knowledge.  Learning  strategies,  unlike  instructional  strategies,  are  learner- 
controlled  as  well  as  learner-generated.  They  engage  the  learners  and  help  them  to  construct 
meaningful  representations,  and  their  success  depends  upon  the  learner  taking  an  active  role  in 
controlling  their  use. 
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Instructional  and  Learning  Strategies  for  Facilitating  Structural  Knowledge 


There  are  a  number  of  instructional  strategies  for  conveying  structural  knowledge 
representations  and  therefore  affecting  the  way  that  learners  encode  these  structures  into 
memory.  There  are  also  a  number  of  learning  strategies  that  learners  can  use  to  build  their  own 
structural  knowledge  representations  (Jonassen,  Beissner,  &  Yacci,  in  press).  Instructional 
strategies  include  explicit  graphic,  mapping  techniques,  such  as  graphical  organizers,  spider 
maps,  semantic  maps,  and  causal  interaction  maps,  for  conveying  knowledge  structures  to 
students.  Teachers  and  designers  may  use  a  number  of  verbal  instructional  techniques,  such  as 
content  structures  and  elaboration  theory,  to  convey  the  underlying  structures  in  materials  to 
students.  A  number  of  graphical  learning  strategies,  such  as  networking  and  pattern  noting,  may 
be  used  by  learners  to  build  their  own  knowledge  representations.  Among  these  strategies,  one  of 
the  most  effective  is  a  study  strategy  called  the  Frame  Game  that  was  developed  to  accompany  an 
educational  psychology  textbook  (Clifford,  1981).  The  Frame  Game  is  a  text  processing  strategy 
that  identifies  the  most  important  concepts  in  a  textbook  chapter  and  then  requires  the  learner  to 
identify  the  relationships  between  the  concepts  by  assigning  them  to  predetermined,  mapped 
relationships  (see  Figure  2).  This  analysis  strategy  requires  that  learners  search,  contrast, 
validate,  elaborate,  confirm,  and  test  information  from  the  chapter.  These  relationship  maps  may 
be  used  to  engage  learners  in  generative  processing  of  textual  information  or  to  depict  the 
information.  In  this  study,  we  compare  the  provision  of  structural  information  to  learners  as  a 
review  of  the  information  with  the  learner  analysis  of  the  relationships  between  the  concepts  in  the 
chapter. 

The  arrangements  of  boxes  in  Figure  2  represent  different  relationships  between  concepts,  such  as 
hierarchical,  sequential,  and  associates  relationships.  Students  have  to  analyze  the  concepts  and 
decide  which  combinations  can  fit  into  the  structures.  Such  an  activity  requires  deep  level 
semantic  analysis  of  the  main  ideas  in  each  book  chapter. 

Purpose  of  Study 

The  primary  purpose  of  this  study  was  to  compare  the  effects  of  a  structural  knowledge 
instructional  strategy  with  the  effects  use  of  a  structural  knowledge  learning  strategy  on  the 
acquisition  of  structural  knowledge.  Specifically,  we  wanted  to  compare  the  effects  of  providing 
graphical  organizers  in  the  form  of  completed  Frames  with  requiring  students  to  complete  Frames 
as  a  study  strategy  prior  to  examinations. 

Method 

Subjects 

Fifty-six  students  from  two  sections  of  a  General  Psychology  (Psy  101)  course  at  a  large 
community  college  in  metropolitan  Denver  completed  the  study.  Ten  other  students  failed  to 
complete  all  of  the  tests  so  were  dropped  from  the  analysis.  There  were  27  subjects  in  the  Group  1, 
and  28  in  Group  2. 

Instruments 

Three  subject-matter  exams  were  designed  for  the  experiment.  Each  exam  consisted  of  60 
questions  worth  one  point  each,  37  of  which  were  multiple-choice  questions  testing  recall  and 
comprehension  of  the  text  and  lecture  material.  In  order  to  assess  structural  knowledge 
acquisition  following  various  treatments,  we  developed  three  subscales  to  measure  different 
aspects  of  structural  knowledge:  a)  10  relationship  proximity  judgements,  b)  8  semantic 
relationships,  and  c)  5  analogies.  All  of  the  structural  knowledge  test  questions  were  developed  to 
focus  on  relationships  between  important  concepts  contained  in  the  textbook  chapters. 

The  relationship  proximity  judgments  required  that  students  judge  the  strength  of  the 
relationship  between  two  terms  and  assign  a  number  between  1  and  9  to  each  of  several  pairs  of 
concepts  to  indicate  how  strong  a  relationship  they  thought  existed  between  the  concepts  in  each  pair 
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(Diekhoff,  1983;  Schvaneveldt,  Durso,  Goldsmith,  Breen,  Cooke,  Tucker  &  DeMaio,  1985; 
Shavelson,  1972).  For  example: 


  2,  endorphins  -  amygdala 

  3.  cerebral  cortex  —  Broca's  area 

  4.  seratonin  -  temporal  lobe 

The  semantic  relationships  subscale  consisted  of  eight  multiple  choice  questions  that 
required  students  to  understand  and  recognize  the  nature  of  the  relationship  between  two  concepts, 
These  relationships  were  paraphrased  from  the  text  book,  For  example: 

  16.  sensory  registers  ....  short  term  memory 

a.  precedes 

b.  is  defined  by 

c.  is  independent  of 

d.  is  inferred  by 

  17.  acoustic  ....  memory 

a.  results  in 

b.  is  independent  of 

c.  is  opposite  of 

d.  is  an  example  of 

Finally,  the  analogies  subscale  required  students  to  complete  5  analogies  consisting  of 
four  of  the  concepts  from  the  hypertext.  For  example: 

  22.  decay  :  forgetting  ::    :  retrieval 

a.  mood 

b.  repression 

c.  rehearsal 

d.  primacy 

  23.  acquisition  :  extinction  ::  punishment  :   

a.  negative  reinforcement 

b.  positive  reinforcement 

c.  variable  reinforcement 

d.  classical  conditioning 

These  questions  were  used  to  assess  structural  knowledge  acquisition.  In  order  to  provide 
standards  for  assessment,  the  researchers  agreed  on  the  answers  to  each  of  these  questions. 
Answers  to  the  multiple  choice  questions  were  recorded  on  the  front  of  Scantron  answer  sheets; 
semantic-proximity  items  were  written  on  the  back.  Exams  covered  history  and  methodology  of 
psychology  (chapters  1  &  2),  physiology,  sensation  and  perception  (chapters  3  &  4),  and  learning 
and  memory  (chapters  6  &  7). 
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Materials 

A  list  of  researcher-generated  key  concepts  and  terms  was  generated  for  each  chapter  to  be 
used  as  student  study  aids;  terms  were  drawn  from  the  textbook,  instructor  notes  and 
examinations  during  the  previous  semester.  Researcher-generated  study  maps  were  generated 
for  chapters  3,  4,  6,  and  7;  mapping  allowed  students  to  visualize  the  relationships  among  terms  — 
superordinate  (hierarchical)  and  sequential  organization,  as  well  as  the  depiction  of  the  terms  in 
classes,  analogies,  similarities,  cause  and  effect,  and  opposites  (see  Figure  2).  Templates  of  maps 
were  designed  for  a  student-generated-mapping  exercise  to  be  assigned  prior  to  Exam  3.  These 
maps  were  the  same  as  the  instructor  provided  maps  in  Figure  2  except  that  most  of  the  locations  in 
the  maps  were  empty,  requiring  the  student  to  fill  in  the  appropriate  concepts. 
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Figure  2 

Procedure 

The  study  was  a  quasi-experimental  design  using  intact  psychology  classes.  The  classes 
met  three  times  a  week  at  10:00  (Group  1)  and  1;00  PM  (Group  2).  One  of  the  researchers  served  as 
instructor  for  both  sections  of  the  course. 
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A  counterbalanced  design  was  used  to  control  for  practice  effect  and  chapter  difficulty. 


Group  1 

Group2 

Exam  1:  Baseline 

Chapters  1, 2 

Chapters  1, 2 

Exam  2:  Maps  Provided 

Chapters  3, 4 

Chapters  6, 7 

Exam  3:  Maps  Generated 

Chapters  6, 7 

Chapters  3, 4 

One  week  prior  to  Exam  1,  a  study  sheet  containing  the  key  terms  from  lecture  and  text  was 
distributed.  Exam  1  was  administered  to  both  groups  at  the  beginning  of  the  4th  week  of  class.  It 
covered  chapters  1  and  2  (history  &  methodology)  and  served  as  the  within-group  control  measure. 

At  the  end  of  the  4th  week  (2  sessions  after  Exam  1)  the  investigators  informed  the  students 
that  they  would  be  involved  in  a  study  about  using  learning  strategies  that  would  help  them 
improve  their  learn  skills. 

One  week  prior  to  Exam  2,  a  list  of  key  terms  and  an  instructor-generated  map  of  terms 
(graphical  organizer,  Figure  3)  was  distributed  to  students;  30  minutes  of  class  time  was  devoted  to 
explaining  the  mapping  rationale.  Exam  2  was  administered  at  the  beginning  of  the  8th  week  of 
class.  Group  1  was  tested  on  chapters  3  and  4  (physiology,  sensation  &  perception);  Group  2  was 
tested  on  chapters  6  and  7  (learning  &  memory),  During  the  class  period  following  the  evam,  the 
instructor  reviewed  the  exam  and  administered  a  questionnaire  assessing  the  extent  of  use  and 
helpfulness  of  the  instructor-provided  maps. 

A  week  prior  to  Exam  3,  a  list  of  key  terms  and  a  student-generated  mapping  exercise  were 
distributed  to  students;  30  minutes  of  class  time  was  devoted  to  the  procedure  of  generating  maps. 
Students  were  assigned  to  complete  the  maps  for  homework.  The  following  class  period,  the 
student-generated  maps  were  collected.  They  were  evaluated  on  a  9-point  Likert  scale  and 
returned  at  the  next  class,  which  was  devoted  to  review  and  to  helping  students  complete  the  maps  of 
difficult  items.  Then  students  were  given  copies  of  the  same  maps  which  had  been  distributed  to 
their  counterparts  for  Exam  2.  Exam  3  was  administered  at  the  start  of  the  12th  week  of  classes. 
Group  1  was  tested  on  ?hapters  6  and  7  (learning  &  memory);  Group  2  was  tested  on  chapters  3  and 
4  (physiology,  sensation  &  perception).  During  the  class  period  following  the  exam,  the  instructor 
reviewed  the  exam  and  administered  a  questionnaire  assessing  the  extent  of  use  and  helpfulness 
of  the  student-generated  and  instructor-provided  maps. 
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Mean  Proximity  Correlations  by  Exam 
Figure  2 

Results 

The  semantic  proximity  scores  for  each  of  the  students  completing  all  three  exams  were 
correlated  with  the  expert  ratings,  yielding  a  correlation  coefficient  for  each  subject.  These 
coefficients  and  the  relationship  scores  (0-8),  analogy  scores  (0-5),  and  the  recall/comprehension 
items  (0-37)  were  all  analyzed  using  a  repeated  measures  analysis  of  variance  (ANOVA).  The 
means  for  the  proximity  scores  by  group  are  illustrated  in  Figure  3,  and  the  relationship  and 
analogy  scores  are  illustrated  in  Figure  4. 
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Fiaure  3 

Relationship  and  Analogy  Scores  by  Exam 

The  ANOVA  calculated  on  the  proximity  correlations  indicated  that  Group  1  significantly 
outperformed  Group  2  (Table  1).  This  was  especially  true  on  the  first  and  second  exams.  Note  that 
this  question  type  was  new  to  students  on  the  first  exam.  Howeve,  a  significant  interaction 
occurred  between  the  second  and  third  exams.  Group  1  appeared  to  be  adversely  affected  by  the 
generative  learning  strategy  which  appears  to  have  significantly  benefitted  Group  2  students, 


Source :  df :        Sum  of  Squares ;   Mean  Square :     F-test :  P  yalue : 


Group  (A) 

1 

.601 

.601 

4.432 

.0402 

subjects  w.  groups 

51 

6.915 

.136 

Repeated  Measure  (B) 

2 

.348 

.174 

2.274 

.1081 

AB 

2 

.72 

.36 

4.713 

.011 

B  x  subjects  w.  groups 

102 

7.797 

.076 

Table  1 

ANOVA  on  Mean  Proximity  Correlations 

On  the  relationship  scores,  no  main  effect  for  group  occurred;  however,  a  significant  main 
effect  occurred  for  the  repeated  measures  (Table  2).  It  is  obvious  from  Figure  4  that  both  groups 
improved  their  relationship  knowledge  by  focusing  on  the  relationships  when  using  the  instructor- 
provided  graphic  organizers.  Structural  knowledge  acquisition  improved  significantly.  Between 
the  second  and  third  exams,  an  apparent  task-by-group  interaction  occurred  (p<.05).  Group  1 
relationship  scores  appear  to  have  benefitted  more  from  the  learning  strategy  than  Group  2  scores. 
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Source : 

df: 

Sum  of  Squares : 

Mean  Square : 

F-test: 

P  value : 

Group  (A) 

1 

.183 

.183 

.074 

.7865 

subjects  w.  groups 

51 

126.043 

2.471 

Repeated  Measure  (B) 

2 

414.503 

207.252 

128.225 

.0001 

AB 

2 

13.966 

6.983 

4.32 

.0158 

B  x  subjects  w.  groups 

102 

164.864 

1.616 

Table  2 

ANOVA  on  Relationship  Scores 

The  analogy  scores  changed  significantly  across  exams.  Group  2  increased,  then 
decreased,  while  Group  1  decreased  then  increased.  A  similar  interaction  effect  occurred  on  the 
analogy  scores  (Table  3).  Group  1  scores  improved  significantly  from  the  second  to  the  third 
exam. 


Source : 

df: 

Sum  of  Squares : 

Mean  Square : 

F-test: 

'  value : 

Group  (A) 

1 

1.484 

1.484 

.812 

.3719 

subjects  w.  groups 

51 

93.245 

1.828 

Repeated  Measure  (B) 

2 

20.994 

10.497 

13.082 

.0001 

AB 

2 

17.162 

8.581 

10.694 

.0001 

B  x  subjects  v.  groups 

102 

81 .844 

.802 

Table  3 
ANOVA  on  Analogy  Scores 

In  attempting  to  explain  the  effects  that  occurred  in  the  study,  all  of  the  possible  factors  need 
to  be  analyzed.  The  possible  causal  factors  include  the  treatment  (organizer  vs.  learning 
strategy),  the  content  or  test  difficulty,  and  individual  differences  between  the  groups.  There  was 
no  significant  difference  between  the  groups  in  terms  of  their  preferences  for  structuring 
materials  (field  independence)  as  measured  by  the  Hidden  Figures  Test  (Ekstrom,  French  & 
Harmon,  1963).  There  was  good  reason  to  believe  that  content  difficulty  may  be  a  factor.  Group  1 
experienced  more  difficulties  on  the  second  exam  covering  chapters  3  and  4  while  Group  2 
experienced  similar  difficulties  on  the  third  exam  which  covered  the  same  chapters. 

In  order  to  test  this  hypothesis,  we  examined  the  total  exam  scores  including  all  of  the 
subscales.  Total  scores  on  the  exams  increased  proportionately  for  both  groups  from  the  first  to  the 
second  exam;  however,  Group  2  scores  decreased  significantly  on  the  third  exam.  This  would  tend 
to  discount  the  content  difficulty  argument.  No  significant  differences  in  the 
recall/comprehension  scores  occurred.  Since  the  strategies  focused  on  the  acquisition  of  structural 
knowledge,  this  finding  was  expected.  The  interaction  between  group  and  repeated  measures  was 
significant  however  (F=4.06,  p<.05).  The  performance  for  Group  1  was  consistent  across  exams 
while  Group  2  scores  decreased  significantly  on  the  third  exam.  This  would  indicate  that  either 
the  test  and/or  the  material  was  more  difficult  overall  or  that  recall  was  adversely  affected  by  the 
learning  strategy. 

At  the  end  of  the  seester,  students  completed  a  recall/comprehension  examination  over  the 
entire  semester's  material.  Each  of  the  items  in  the  exam  were  classified  by  the  exam  (and 
therefore  the  treatment)  they  were  derived  from.  These  questions  comprise  a  recall/retention  test 
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of  the  content  learned  under  each  of  the  treatments.  The  retention  scores  showed  the  same  pattern 
as  all  of  the  structural  knowledge  questons  in  the  chpater  tests.  A  repeated  masure  ANOVA 
showed  no  main  effects  but  a  significant  group-by-treatment  inter  jtion  (F=17.65,  p<.001). 
Retention  scores  were  equivalent  for  maerials  from  the  first  exam,  Group  1  decline  on  material 
from  the  second  exam  (Chapters  3  and  4,  M=7.7)  and  improved  on  material  from  the  third 
(Chapters  6  and  7,  M=10.2)  while  Group  2  had  the  opposite  effect  (M=10.0  and  7.4).  Clearly,  the 
material  from  chhapters  3  and  4  was  more  difficult  than  that  from  chapters  6  and  7. 

Discussion 

A  number  of  factors  seem  to  have  affected  the  learning  of  material  in  this  experiment.  One 
of  the  most  obvious  is  the  difficulty  of  the  material.  Clearly  the  material  in  Chapters  3  and  4 
(physiology,  sensation,  and  perception)  was  more  difficult  than  that  from  Chapters  6  and  7.  The 
former  discussion  was  more  technical  and  less  related  to  the  student's  prior  knowledge.  It  is 
easier  to  relate  learning  and  memory  concepts  to  a  class  of  college  students. 

However,  difficulty  of  the  material  was  not  the  only  cause  of  the  results.  Since 
immmediate  recall  and  comprehension  performance  was  not  appear  to  be  affected  by  the  content 
for  Group  1.  Their  recall  scores  were  essentially  equal.  Group  2  however  was  more  affected.  In 
addition  to  content  difficulty,  their  decreased  performance  was  also  caused  in  part  by  motivational 
deficiencies.  Group  1  examination  performances  tended  to  exceed  that  of  Group  2.  This  might  be 
explained  by  any  of  several  factors,  or  a  combination  thereof.  First  is  intrinsic  motivation.  At 
this  college,  afternoon  classes  (Group  2)  are  generally  more  lethargic  and  less  academically 
motivated  than  morning  classes;  frequently  the  students  in  the  afternoon  classes  enroll  just  before 
the  term  begins  (attending  college  is  more  of  an  afterthought).  Second  is  extrinsic  motivation. 
Although  students  were  asked  to  complete  the  mapping  exercise,  they  did  not  receive  any  course 
credit  for  doing  so.  13  of  27  students  in  Group  1,  but  only  9  of  28  students  in  Group  2  mapped  both 
chapters  for  the  third  exam.  Future  stu  dies  should  assign  course  credit  based  on  the  quality  of 
maps  generated.  Another  issue  related  to  extrinsic  motivation  is  the  fact  that  the  instructor  curved 
the  students'  grades  on  each  exam.  Thus,  the  students  in  Group  1,  which  scored  higher  than  Group 
2,  were  forced  to  master  more  of  the  material  if  they  wanted  to  earn  high  grades.  In  future  studies, 
grades  should  not  be  curved. 

Another  issue  relates  to  the  difficulty  of  the  mapping  exercise.  The  exercise  may  have 
placed  too  great  a  cognitive  load  on  students.  The  30  minutes  instruction  in  completing  the  maps 
was  probably  inadequate;  we  believe  that  future  3tudies  should  provide  practice  in  generating 
maps  of  familiar  material  before  students  are  asked  to  generate  maps  of  new  material. 
Furthermore,  they  probably  needed  more  scaffolding  in  completing  such  exercises.  For  example, 
the  list  of  key  concepts  should  contain  only  the  terms  which  can  actually  be  used  to  complete  the 
maps;  in  this  study,  students  received  a  list  of  all  of  the  concepts  they  were  responsible  for,  but 
many  of  those  were  not  addressed  by  the  maps.  Since  it  appears  that  students  acquired  more  benefit 
from  being  provided  with  completed  maps  than  having  to  generate  their  own  (these  maps  provided 
scaffolding  for  relating  and  remembering  the  material)  and  generating  maps  was  a  new 
learning  strategy  for  the  students,  they  did  not  have  time  to  learn  how  to  incorporate  this  study 
strategy  in  just  a  few  days;  Duffy  and  Roehler  (1989)  suggest  that  it  takes  months  for  a  new 
learning  strategy  to  displace  a  well-established  old  one.  To  give  students  time  to  incorporate  the 
strategy  into  their  repertoires,  future  studies  should  utilize  a  single  strategy  for  an  entire 
semester. 

Students1  reactions  to  the  two  strategies  also  suggests  the  need  for  further  investigation.  At 
least  77%  of  the  students  in  each  class  said  they  found  the  maps  to  be  "somewhat  helpful"  or  "very 
helpful  study  aids.  But  as  Clark  (1982)  found  in  an  earlier  study,  there  appears  to  be  a 
achievement  and  enjoyment;  67%  of  the  survey  respondents  in  Group  1  (44%  of  the  total  group) 
preferred  being  given  maps,  while  64%  in  Group  2  (32%  of  the  total  group)  preferred  having  to 
construct  maps.  Most  of  the  students  who  preferred  being  given  maps  commented  that  having  to 
complete  the  maps  forced  them  to  pay  closer  attention  to  the  material.  Whether  students 
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preferences  and  achievement  would  change  with  less  difficult  mapping  exercises  and  more 
training  in  mapping  must  be  studied. 

Another  question  relating  to  the  use  of  the  maps  is  the  nature  of  the  cognitive  processing 
students  used.  Did  they  engage  in  rote  memorization  or  perhaps  some  form  of  verbal  mediation 
such  as  self-explanations  or  comparative  elaborations?  Future  studies  should  include  debriefing 
sessions  to  try  to  identify  the  types  of  processing  students  used. 

Students1  performances  on  the  relationship  items  suggests  that  verbal  competency  might 
have  played  a  role.  During  the  exams,  several  students  asked  the  instructor  the  definition  of 
words  (e.g.,  "exemplifies1').  Future  studies  should  assess  students1  verbal  competency  to 
determine  if  it  influences  their  performance  on  the  examinations. 

Finally  is  the  issue  of  content  difficulty.  For  the  total  scores  there  was  no  group  by  test 
interaction,  but  for  the  structural-knowledge  items  there  was.  This  suggests  that  there  was  an 
interaction  for  content  by  cognitive  task  difficulty.  That  is,  it  may  have  been  more  difficult  to 
assimilate  structural  knowledge  for  chapters  3  and/or  4  than  for  chapters  6  and  7.  Chapter  4  did  not 
have  a  single  organizing  structure;  thus  no  hierarchical  map  was  provided  for  that  chapter,  the 
lack  of  such  a  structure  may  have  contributed  to  the  depressed  structural  knowledge  scores  on  that 
examination.  Further  analysis  of  the  content  of  test  items  for  chapters  3  and  4  is  needed. 
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Abstract 

Hypertext  researchers  and  designers  contend  that  the  hypertext  information  structures  may  reflect  the 
semantic  structures  of  human  memory.  Further,  they  believe  that  mapping  the  semantic  structure  of 
an  expert  onto  hypertext  information  structure  and  explicitly  illustrating  that  structure  in  the 
hypertext  will  result  in  improved  comprehension  because  the  knowledge  structures  of  the  users  will 
reflect  the  knowledge  structures  of  the  expert  to  a  greater  degree  (Jonassen,  1990,  1991b).  This  reviews 
techniques  for  ascertaining  an  expert's  knowledge  structure  and  mapping  it  onto  hypertext.  It  then 
reviews  the  results  of  three  studies  that  assess  the  effects  of  different  methods  for  explicitly  mapping 
expert  knowledge  structures  onto  hypertext  on  the  acquisition  of  structural  knowledge  in  the  learners. 
The  studies  show  that  depicting  knowledge  structures  in  the  form  of  a  graphical  browser  or  by  making 
explicit  the  structural  nature  of  the  links  during  traversal  does  not  improve  learners'  acquisition  of 
structural  knowledge.  However,  when  assigned  the  task  of  generating  a  semantic  network  following 
browsing,  structural  knowledge  acquisition  improved  significantly.  It  is  theassigned  processing 
task  and  goals  for  learning  while  'interacting  with  a  hypertext  that  appears  to  most  significantly 
determine  the  effects  of  its  use  on  learners'  knowledge  structures. 
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INTRODUCTION 

Associative  Hypertext  Structures 

Hypertext  researchers  have  recently  asserted  that  hypertext  mimics  the  associative  networks 
of  human  memory  (Fiderio,  1988;  Jonassen,  1990, 1991a,  1991b).  Bush  (1945),  who  is  credited  with 
developing  the  first  prototype  hypertext  system,  observed  that  ideas  result  from  "tVe  association  of 
thoughts,  in  accordance  with  some  intricate  web  of  trails  carried  by  cells  of  the  brain".  This 
observation  and  the  continued  development  of  associative  network  theory  have  provided  a  conceptual 
foundation  for  the  development  of  hypertexts. 

Hypertext,  like  other  technologies  such  as  databases  and  expert  systems,  is  a  knowledge-based 
system.  That  is,  subject  content  is  stored  in  a  knowledge  base  which  is  structured  by  a  particular  data 
model,  which  defines  the  organization  of  the  information  contained  in  the  knowledge  base.  This 
organization,  in  turn,  defines  the  logical  relationships  between  the  content  units  in  the  knowledge 
base.  The  logic  in  each  type  of  data  model  varies  with  the  kinds  of  relationships  that  comprise  it.  A 
hypertext  engine,  on  the  other  hand,  is  associative.  That  is,  it  is  based  upon  an  associative  network  of 
ideas.  Links  may  be  defined  by  an  open  set  of  associations,  so  hypertext  structures  may  take  on  a 
variety  of  forms.  Therefore,  the  structure  of  any  hypertext  may  emulate  the  logical  structures  of  a 
variety  of  instructional  designs  or  functions  (Jonassen,  1991b). 

The  belief  that  hypertext  can  mimic  human  associative  networks  implies  that  an  appropriate 
method  for  structuring  hypertext  is  to  mirror  the  semantic  network  of  an  experienced  or 
knowledgeable  performer  or  expert.  Semantic  networks  are  rooted  in  schema  theory  (Rumelhart  & 
Ortony,  1977).  Personal  knowledge  is  stored  in  information  packets  or  schemas  that  comprise  our 
mental  constructs  for  ideas.  Schemas  have  attributes,  which  they  share  with  other  schemas.  Each 
schema  that  we  construct  represents  a  mini-framework  in  which  to  interrelate  elements  or  attributes 
of  information  about  a  topic  into  a  single  conceptual  unit  (Norman,  Gentner,  &  Stevens,  1976).  These 
mini-frameworks  are  organized  by  the  individual  into  a  larger  network  of  interrelated  constructs 
known  as  a  semantic  network.  These  networks  are  composed  of  nodes  (representations  of  schemas) 
and  ordered  labelled  relationships  that  define  the  prepositional  relationships  between  them  (Norman, 
Gentner  &  Stevens,  1976). 
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The  general  hypotheses  for  these  studies  i?  *hit  mapping  the  semantic  network  of  an  expert  or 
knowledgeable  person  onto  the  structure  of  a  hypertext  will  contribute  to  the  development  of  the 
learners1  knowledge  structures  while  using  the  hypertext  to  learn. 

Learning  consists  of  building  new  knowledge  structures  by  assimilating  environmental 
information  and  constructing  new  nodes  that  describe  and  interrelate  them  with  existing  nodes  and 
with  each  other  (Norman,  1976).  The  interrelated  knowledge  reflected  in  semantic  networks  enables 
humans  to  combine  ideas,  infer,  extrapolate  or  otherwise  reason  with  the  information.  Learning 
requires  the  formation  of  links  between  existing  knowledge  and  new  knowledge  in  order  to 
comprehend  information  from  the  environment.  Learning  therefore  may  be  conceived  of  as  a 
reorganization  of  the  learner's  knowledge  structure  that  results  from  the  learner's  interactions  with 
the  environment.  However,  learning  is  also  influenced  by  intentional  instruction.  Research  has 
shown  that  as  a  result  of  teachinr  learners'  knowledge  structures  more  closely  resemble  the 
instructor's  (Shavelson,  1974;  Thro,  1978),  that  is,  learners  acquire  the  teachers  knowledge  structure, 
which  in  turn  improves  comprehension.  So,  learning  is  the  linking  of  new  information  with 
exisiting  knowlege,  a  process  which  is  influenced  by  the  mapping  of  the  teacher's  or  expert's  subject 
matter  knowledge  onto  the  learner's  knowledge  structure.  This  conception  provides  the  rationale  for 
mapping  the  expert's  knowledge  structure  onto  learning  environments,  such  as  hypertext. 

Hypertext  engines  or  structures  can  be  designed  to  reflect  the  semantic  structure  of  a  subject 
matter  expert.  The  assumption  for  doing  so  is,  "if  the  node-link  structure  of  the  hypertext  reflects  the 
semantic  structure  of  the  expert,  the  expert's  knowledge  structure  may  be  more  effectively  mapped  onto 
the  novice  browser,  thereby  improving  comprehension."  This  is  the  assumption  that  has  been 
investigated  in  the  studies  described  below. 

EXPERIMENT  1 

The  most  direct  way  to  map  the  expert's  semantic  structure  onto  a  hypertext  is  to  use  the  expert's 
semantic  map  as  a  graphical  browser  in  the  hypertext.  Graphical  browsers  are  maps  or  graphical 
listings  of  available  nodes  in  a  hypertext.  They  represent  a  graphical  interface  between  the  user  and  a 
hypertext  that  is  designed  to  reduce  navigation  problems  within  the  hypertext  (Jonassen,  1988). 
Getting  lost  in  a  large  web  of  hypertext  nodes  and  links  is  a  commonly  reported  problem  among 
hypertext  users,  so  graphical  browsers  are  used  to  provide  a  spatial  orientation  to  the  available  nodes 
in  a  hypertext.  When  arranging  the  nodes  in  a  graphical  browser  according  to  an  expert's  semantic 
map,  the  hypertext  is  explicitly  conveying  the  organization  of  ideas  in  the  expert's  knowledge 
structure.  So,  while  navigating  through  a  hypertext,  the  user  is  in  effect  navigating  through  the 
expert's  knowledge  structure.  The  research  questions  in  this  study  focus  on  the  extent  to  which  the 
semantic  structure  illustrated  in  graphical  browsers  actually  maps  onto  the  user's  knowledge 
structure.  To  what  extent  will  the  user  model  or  replicate  that  structure  in  their  own  knowledge 
representations? 


Method 

Sample 

This  study  involved   undergraduate  pre-service  teachers  in  a  teacher  education  program 
Instruments 

In  order  to  assess  the  effects  of  semantically  structured  hypertext  on  learner's  knowledge 
structures,  it  was  necessary  to  develop  instruments  that  assessed  the  learners'  structural  knowledge. 
Structural  knowledge  is  the  knowledge  of  how  concepts  within  a  domain  are  interrelated  (DiekhofF, 
1983).  Structural  knowledge  enables  learners  to  form  the  connections  that  they  need  to  describe  and 
use  scripts  or  complex  schemas.  It  is  a  form  of  conceptual  knowledge  that  mediates  the  translation  of 
declarative  knowledge  into  procedural  knowledge. 

In  order  to  assess  structural  knowledge  acquisition  following  various  treatments,  we 
developed  three  subscales  of  ten  questions  each  to  measure  different  aspects  of  structural  knowledge: 
a)  relationship  proximity  judgements,  b)  semantic  relationships,  and  c)  analogies.  All  of  the 
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structural  knowledge  test  questions  were  developed  to  focus  on  relationships  between  important 
concepts  contained  in  the  hypertext. 

The  relationship  proximity  judgements  required  that  students  assign  a  number  between  1  and 
9  to  each  of  several  pairs  of  concepts  to  indicate  how  strong  a  relationship  they  thought  existed  between 
the  concepts  in  each  pair  (Diekhoff,  1983;  Schvaneveldt,  Durso,  Goldsmith,  Breen,  Cooke,  Tucker  & 
DeMaio,  1985;  Shavelson,  1974),  l  or  example: 

  information  retrieval  systems  and  online  documentation 

  hypertext  processing  strategies  and  database 

The  semantic  relationships  subscale  consisted  of  multiple  choice  questions  that  required 
students  to  identify  the  nature  of  the  relationship  between  two  concepts.  These  relationships  were 
paraphrased  from  the  hypertext  knowledge  base,  For  example: 

  unstructured  hypertext        navigating  hypertext 

a.  produces  problems  in 

b.  defines  the  functions  of 

c.  counteracts  the  effects  of 

d.  enabled  by 

Finally,  the  analogies  subscale  required  students  to  complete  10  analogies  consisting  of  four  of 
the  concepts  from  the  hypertext.  For  example: 

  accessing  information  :  index  ::  integrating  information  :   

a.  links 

b.  hypermaps 

c.  idea  generator 

d.  multi-user  access 

These  questions  were  used  to  assebS  structural  knowledge  acquisition.  In  order  to  provide 
standards  for  assessment,  three  authors  and  researchers  in  the  hypertext  field  agreed  on  the  answers 
to  each  of  these  questions.  In  addition  to  assessing  structural  knowledge,  ten  lower-order 
information-recall  questions  were  developed. 

Materials 

The  hypertext  that  was  used  was  the  HyperCard  version  (Jonassen,  Roebuck  &  Wang,  1990)  of 
the  book  Hypertext  I  Hypermedia  (Jonassen,  1989a).  This  hypertext  is  a  browsing  system  consisting  of 
240  cards  and  1167  links  in  three  stacks  supported  by  bookmarking  and  limited  annotation 
capabilities.  All  treatments  in  all  studies  contained  embedded  referential  links  in  the  cards.  Terms 
in  the  text  were  highlighted,  enabling  learners  to  immediately  traverse  the  links.  Tho  treatments 
varied  in  terms  of  the  types  of  browsers  that  were  made  available  and  the  ways  that  they  depicted 
structural  information.  Each  of  the  75  major  concept  nodes  contained  a  main  "related  terms'1  card, 
which  was  the  first  card  accessed  when  traversing  a  link  to  that  node.  In  the  control  group,  this  card 
provided  a  list  of  term?  that  are  related  to  the  concept  being  currently  examined.  This  list  provided 
links  but  no  structural  information  about  the  nodes  or  links.  The  experimental  treatments  (described 
in  more  detail  below)  replaced  these  lists  with  graphical  browsers  or  retained  the  lists  but  overtly 
communicated  the  nature  of  each  link  when  it  was  being  traversed.  They  were  to  designed  to  provide 
or  require  the  learner  to  generate  explicit  structural  information  about  the  nodes  and  links  in  the 
hypertext. 

In  this  study,  we  compared  the  extent  to  which  users  acquire  structural  knowledge  by  using  a 
graphical  browser  vs.  a  pop-up  window  mediating  each  link.  Each  graphical  browser  showed  the 
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current  node  and  all  nodes  linked  directly  to  it  and  explicitly  depicted  the  nature  of  the  relationship 
between  each  of  those  nodes  (see  Figure  1  for  an  example).  Each  of  the  icons  in  the  graphical  browsers 
were  hot  buttons  which  the  learners  used  to  navigate  through  the  hypertext.  In  order  to  navigate  the 
hypertext,  users  would  click  on  any  icon  to  traverse  the  link  to  that  topic. 

In  the  second  experimental  treatment,  the  same  nodes  related  to  the  current  node  were 
presented  in  list  form  with  no  indication  of  the  nature  of  the  relationship  between  the  nodes.  In  order  to 
traverse  a  link,  users  simply  clicked  on  a  term.  Whenever  the  user  clicked  on  a  term  to  select  it,  that 
is  to  navigate  that  link,  a  window  filling  70%  of  the  screen  area  would  pop  up.  The  text  in  this  pop-up 
window  stated  the  explicit  relationship  between  the  node  the  learner  was  leaving  and  the  node  the 
learner  was  going  to.  So,  whenever  the  student  traversed  a  link  between  the  existing  and  the  target 
node,  a  pop-up  window  stated 


Applications 
of  Hypertext 


Synthesizing 
Information 


•aid  for 


for  example 


COGNITIVE 
LEARNING 
TOOL 

^important 
^components 
of 


learning  theory  rationales  for 


Active 
Structural 
Networks 


Figure  1 
Graphical  Browser 

explicitly  the  relationship  between  the  node  they  were  leaving  and  the  one  they  had  selected,  providing 
an  explicit  statement  of  structural  knowledge. 

Both  experimental  conditions  provided  structural  information  to  the  student.  The  graphical 
browser  treatment  illustrated  the  organization  of  information  in  the  hypertext  knowledge  base,  while 
the  pop-up  treatment  explicitly  depicted  relationships  on  a  node-by-node  basis.  In  order  to  navigate  the 
hypertext,  the  control  group  students  were  provided  the  lists  of  of  related  terms  with  no  indication  of  the 
nature  of  the  relationship  between  the  nodes.  In  order  to  traverse  a  link,  users  simply  clicked  on  a 
term. 


Procedure 

Students  were  given  the  task  of  learning  about  an  important  new  instructional  technology, 
hypertext,  as  an  assignment  in  a  pre-service  instructional  technology  course  by  using  a  hypertext  In 
a  Macintosh  laboratory,  students  individually  interacted  with  and  studied  the  hypertext  in  order  to 
acquire  as  much  information  about  this  new  technology  as  possible.  A  monitoring  program  was 
added  to  the  stacks  to  audit  the  action  and  the  time  allotted  to  each  of  the  student's  actions.  Following 
the  interaction,  learners  were  asked  to  complete  the  posttests 

Results 

The  dependent  variables  in  the  cognitive  domain  in  this  study  were  the  recall,  relationship, 
proximity,  and  analogy  subscales  of  the  posttest.  The  only  independent  variable  was  the  version  of  the 
treatment  administered  to  each  of  the  subjects.  The  recall  (15  items),  relationship  (10  items),  and 
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analogy  (10  items)  subscales  were  totaled  for  each  subject  The  means  are  presented  in  Table  1.  For 
the  proximity  scores,  correlation  coefficients  were  calculated  for  each  subject,  in  which  the  proximity 
ratings  of  the  subject  were  correlated  to  the  proximity  ratings  of  a  panel  of  three  experts,  The  means 
for  these  coefficients  by  group  are  presented  in  Table  1. 

Recall  Proximity     Relationship  Proximity  Analogy 

 Time  fitwio.  Cards 

Treatment 

Control     7.8  (2.1)  .295  (.2)       5.4  (2.3)  ,572  (.08)      3,0  (1.3)  57.2  27.5 

Pop-up     7.4  (2.1)  .333  (.2)       5.6  (1.7)  -616  (.12)      2,7  (1.9)  50,8  45.9 

Map         6.6  (1,9)  .320  (.2)       5.0  (2.2)  .583  (.08)    3.1  (1.6)  51,2  53,2 

Table  1 

 Means  (Standard  Deviations)  of  Dependent  Variables  by  Group  

Table  1 

Means  (Standard  Deviations)  of  Dependent  Variables  by  Group 

One  way  analyses  of  variance  were  calculated  for  each  of  the  four  dependent  variables.  The 
only  dependent  variable  with  even  a  marginally  significant  difference  was  the  recall  variable 
(F(2,93)=2.75,  p<,07).  A  post  hoc  analysis  using  a  Fisher  LSD  test  showed  that  the  only  groups  which 
were  significantly  different  were  the  control  and  the  map  versions,  with  the  control  group 
outperforming  the  map  version  (LSD=1.15).  The  map  version  produced  the  lowest  scores  overall, 

Time  on  task  was  also  compared  by  group.  Control  group  students  spent  the  longest  amount  of 
time  (Af=57.3  minutes)  while  the  pop-up  group  spent  only  50,7  minutes.  Because  of  the  large  standard 
deviations  in  the  amount  of  time  spent  within  group,  this  difference  was  not  significant. 

The  total  number  of  structurally  related  cards  accessed  by  students  was  also  calculated. 
Structural  cards  were  the  maps  (Map  group)  or  the  lists  in  the  other  experimental  group  and  the  control 
group.  These  cards  contained  the  structural  information  that  defined  the  relationships  between  the 
nodes  on  the  knowledge  base.  The  number  of  structural  cards  accessed  was  different  between  groups 
(F(2,93)=3.75,  p<,05)  while  the  total  number  of  cards  accessed  and  the  total  time  spent  viewing 
structural  cards  did  not  differ  significantly.  The  control  group  accessed  106  cards  while  the  pop-up 
and  map  groups  accessed  85  and  92  cards  respectively. 

The  absence  of  main  effects  for  structural  knowledge  outcomes  did  not  necessarily  mean  that 
no  structural  coding  occurred.  In  order  to  assess  the  role  of  structural  cueing  on  structural 
knowledge,  regression  analyses  were  calculated.  The  total  umber  of  cards  accessed  while  reading  the 
hypertext  was  regressed  onto  each  of  the  dependent  variables.  No  significant  differences  occurred. 
However,  when  the  number  of  structural  cards  accessed  was  regressed  on  the  dependent  variables, 
significant  differences  occurred.  The  number  of  high  level  structural  cards  (main  map  and  eight 
first  level  maps,  eg.  Characteristics  of  Hypertext)  that  were  accessed  by  students  significantly 
predicted  the  relationship  scores  (F=3.7,  p=.05).  The  number  of  structural  cards  accessed  also 
significantly  predicted  the  time  spe^t  (F=5.37,  p<.05).  The  fewer  the  structural  cards  that  were 
accessed,  the  more  time  that  was  spent  by  the  students  reviewing  the  knowledge  base. 

Total  time  spent  interacting  with  structural  cards  was  also  calculated  by  group.  The  time 
spent  on  structural  cards  did  not  significantly  regress  on  any  of  the  dependent  variables.  The 
average  proximity  scores  showed  an  effect  of  time,  although  this  effect  was  not  significant  (p<,10). 

Discussion 

The  lack  of  significant  main  effects  in  performance  indicates  that  the  information  processing 
required  by  browsing  hypertext  overshadowed  the  design  attributes  of  the  interface.  The  control 
treatment,  which  did  not  provide  explicit  structural  information,  produced  the  highest  level  of  recall, 
This  finding  is  consistent  with  research  that  shows  that  without  cueing  or  practice,  learners  tend  to 
recall  micropropositions  more  readily  than  macropropositions. 
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There  were  a  number  of  potential  limitations  of  this  study  that  may  have  obviated  more 
consistent  structural  effects.  First,  the  average  time  spent  interacting  with  this  hypertext  was  less 
than  one  hour  for  virtually  all  subjects.  This  short  treatment  period  did  not  provide  adequate  exposure 
to  the  hypertext  or  its  structural  elements.  In  the  follow-up  study,  incentives  will  be  used  to  increase  the 
length  of  the  interactions  with  the  hypertext. 

In  addition  to  the  limited  exposure  to  the  hypertext,  the  lack  of  hypertext  processing  strategies 
(Jonassen,  1989)  likely  precluded  the  most  effective  use  of  the  technology.  Hypertext,  and  the  greater 
control  of  instruction  that  it  requires,  id  a  novel  form  of  instruction  for  these  learners.  The  more  novel 
the  appearance  of  the  hypertext  (ie.  mapping  version),  the  more  negatively  the  students  reacted  to  it  A 
fair  evaluation  of  learning  from  hypertext  can  only  come  from  hypertext-literate  learners  who  have 
developed  a  useful  set  of  strategies  for  navigating  and  integrating  information  form  hypertext. 

The  novel  form  of  instruction,  hypertext,  was  only  exacerbated  by  the  novel  structural  cues  and 
structural  learning  outcomes.  The  explicit  provision  of  structural  information  is  unusual  to  these 
students.  The  requirement  to  process  that  infonnation  and  integrate  it  into  their  own  knowledge 
structures  was  even  more  novel  to  them,  which  negatively  affected  their  performance  on  the  tasks.  As 
evidenced  by  the  floor  effect  on  the  analogy  questions,  these  students  were  not  accustomed  to  higher 
order,  structural  knowledge  outcomes  from  instruction.  The  poor  performance  of  learners  on  higher 
order  and  structural  knowledge  tasks  washed  out  most  of  the  treatment  effects.  The  relationship  test 
resulted  in  only  one  half  of  the  questions  answered  correctly  and  less  than  ono  third  on  the  analogy 
tests.  Only  one  half  of  the  paraphrased  recall  questions  were  answered  correctly.  Perhaps  verbatim 
recall  questions  may  have  been  better  answered.  This  calls  into  question  the  ability  of  learners  to 
acquire  knowledge  from  hypertext  in  the  context  of  a  learning  assignment. 

A  major  reason  for  the  lack  of  main  effects  in  this  study  as  well  as  the  limited  potential  of 
hypertext  for  learning  is  the  lack  of  generativity  in  the  processing  of  the  information.  Wittrock  (1974) 
claimed  that  meaningful  learning  results  form  generative  processing  in  which  learners  relate  new 
information  to  their  existing  knowledge  in  order  to  accommodate  it  to  ha  they  know  already.  Merely 
attending  to  structural  cues  may  not  engender  generative  processing  of  information.  In  the  next 
study,  the  structural  cues  will  be  compared  with  a  more  generative  process  in  which  the  students  must 
determine  the  nature  of  the  link  relationships  for  themselves,  rather  than  being  informed  by  the 
program. 

The  post  hoc  regression  analyses  provided  some  evidence  that  students  were  in  fact  attending 
to  some  of  the  structural  information.  The  number  of  structural  cards  accessed  appeared  to  predict 
structural  knowledge  acquisition  in  the  form  of  relationship  judgements  in  two  different  forms.  The 
time  spent  with  the  structural  cards  suggested  a  relationship  between  accessing  those  cards  and  the 
level  of  structural  knowledge  acquisition.  These  are  weak  effects  at  best,  though  with  longer  exposure 
to  the  hypertext  and  the  prior  acquisition  of  some  hypertext  processing  strategies  should  clarify  those 
effects. 

Hypertext  is  an  information  retrieval  technology  that  may  have  implications  for  instruction. 
If  it  is  to  fulfill  that  potential,  then  designers  must  verify  methods  for  facilitating  the  learning 

EXPERIMENT  2 

Generative  Processing  of  Structural  Information 

Perhaps  the  most  significant  weakness  of  the  first  study  was  the  lack  of  instructional  support 
provided  he  learner.,  given  the  expectations  of  learning  from  the  hypertext.  Hypertext  is  a  technology 
that  supports  information  retrieval  and  search  tasks.  However,  these  tasks  are  not  necessarily 
predictive  of  learning.  The  most  significant  potential  problem  in  learning  from  hypertext  is 
integration  of  the  information  that  is  browsed  into  the  learners  knowledge  structure  (Jonassen, 
1989a).  Learning  requires  that  users  not  only  access  information  but  also  interpret  it  by  relating  it  to 
prior  knowledge,  as  discussed  in  the  introduction. 
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Method 

Subjects 

112  students  enrolled  in  an  undergraduate  course  in  instructional  technology  at  a  western 
college  participated  in  this  study.  The  students  are  enrolled  in  a  teacher  certification  program. 

Instruments  and  Materials 

The  instruments  used  were  identical  to  Experiment  1.  However,  the  treatments  were  altered. 
Three  treatments  were  developed  for  this  study  using  the  same  HyperCard  version  (Jonassen, 
Roebuck  &  Wang,  1990)  of  the  book  Hypertext/  Hypermedia  (Jonassen,  1989)  that  was  used  in 
Experiment  1.  The  second  study  attempted  to  provide  additional  instructional  support  by  including  a 
more  generative  form  of  treatment.  Generative  learning  occurs  when  learners  relate  information 
meaningfully  to  prior  knowledge  (Wittrock,  1974).  This  study  included  a  generative  processing 
treatment.  Rather  than  being  told  what  the  nature  of  the  relationship  was  between  the  nodes  being 
linked,  the  learners  were  required  to  classify  the  nature  of  each  link  themselves.  The  pop-up  window 
presented  12  different  link  types  and  required  the  learner  to  determine  which  of  the  link  types  most 
accurately  depicted  the  nature  of  the  relationship  implied  by  the  link  that  they  were  traversing.  The 
user  had  the  option  of  returning  to  the  previous  node  or  moving  forward  to  see  the  node  they  had  selected 
as  many  times  as  necessary.  Knowledge  of  results  was  provided  for  each  selection  by  the  user  until 
thb  user  selected  the  correct  link  type. 

The  control  treatment,  consisted  of  embedded  referential  links  in  the  cardsin  addition  to  the 
'related  terms"  nodes  attached  to  the  75  m^jor  concepts.  The  first  experimental  treatment  consisted  of 
the  same  capabilities  with  the  added  feature  of  a  pop-up  window  mediating  each  link,  as  in  Experiment 
1.  Text  in  the  window  stated  the  explicit  relationship  between  the  node  the  learner  was  leaving  and  the 
node  the  learner  was  going  to  whenever  the  student  traversed  a  link  between  the  existing  and  the  target 
node  (an  explicit  statement  of  structural  knowledge).  The  second  experimental  treatment,  the 
generative  treatment,  used  a  pop-up  window  which  asked  the  student  to  classify  the  nature  of  the 
relationship  between  the  node  they  were  leaving  and  the  one  they  were  traversing  to.  Students  were 
provided  the  option  of  reviewing  the  pervious  node  and  the  target  node  in  order  to  help  classify  the 
relationship.  The  first  experimental  condition  provided  structural  information  to  the  student,  and  the 
second  required  the  learner  generate  his/her  own  relational  information. 

Procedure 

Students  were  given  the  task  of  learning  about  an  important  new  instructional  technology, 
hypertext,  as  an  assignment  in  a  pre-service  instructional  technology  course  by  using  a  hypertext.  In 
a  Macintosh  laboratory,  students  individually  interacted  with  and  studied  the  hypertext  in  order  to 
acquire  as  much  information  about  this  new  technology  as  possible.  A  monitoring  program  was 
added  to  the  stacks  to  audit  the  action  and  the  time  allotted  to  each  learner  action.  Following  the  study 
period,  learners  completed  the  posttests. 

Results 

As  in  the  first  study,  the  recall  scores  of  the  control  group  (no  structural  information  provided) 
were  higher  than  eithe;  of  the  two  structural  treatments,  however,  this  difference  only  approached 
significance  (p=.06).  No  differences  between  relationship,  proximity  judgements,  semantic 
relationships,  or  analogy  scores  occured.  As  anticipated,  recall  scores  were  higher  in  the  control 
group,  but  neither  structural  strategy  produced  any  increase  in  structural  knowledge  acquisition. 
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Treatment 

N 

Recall 

Proxi  mity 

Relationship 

Analogy 

uonvroi 

c  \ 

C  TO  /  I  C\ 

-  1  Ad  (  OK\ 

c  i  ft /  i  c\ 

Pop-Up 

11 

4.90(1.4) 

-.088  (.25) 

5.73(1.5) 

2.68  (1.6) 

Generative 

24 

4.42  ((1.2) 

-.055  (.24) 

5.75(1.6) 

2.75(1.5) 

Table  2 

In  this  study,  we  also  assessed  the  individual  difference  characteristics,  field  independence 
(Hidden  Figures  Test)  and  global  reading  ability/vocabulary  (Extended  Range  Vocabulary  Test,  both 
from  the  Kit  of  Factor  Referenced  Cognitive  Tests  (Ekstrom,  French,  &  Harman,  1976)  looking  for 
aptitude-by-treatment  interactions.  Marginally  significant  interactions  occurred  on  the  recall  and 
relationship  variables.  In  the  control  and  generative  treatments,  field  independent  students 
performed  better  on  the  recall  task,  but  they  were  impeded  by  the  pop-up  version.  On  the  semantic 
relationship  task,  the  opposite  occurred.  Field  independent  students  in  the  pop-up  version 
outperformed  field  dependents  while  theregression  equations  for  the  control  and  generative  groups 
were  flat.  No  significant  interactions  occurred  on  the  analogy  task.    Nor  were  there  any  interactions 
of  treatments  with  vocabulary  scores. 

EXPERIMENTS 

While  instructional  tasks  are  normally  thought  to  be  an  important  predictor  of  integration  of 
information  into  knowledge  structures  (although  this  was  not  entirely  evident  fro^n  Experiment  2), 
probably  the  most  important  determinant  of  learning  is  the  awareness  and  understanding  of  the 
required  task  or  learning  outcome.  In  the  previous  two  studies,  it  became  clear  that  learners  were  not 
certain  how  to  "learn  from  hypertext".  Students  in  these  studies  were  most  familiar  with  performing 
specific,  convergent,  recall-oriented  learning  tasks.  The  uncertainty  involved  in  trying  to  integrate 
knowledge  by  focusing  cn  structural  relationships  when  the  task  was  not  clear  became  obvious 
throughout  the  course  of  experimentation.  It  was  also  clear  from  the  first  two  studies  that  students, 
based  upon  their  scores  on  the  relationship  and  analogy  tests,  were  not  familiar  or  competent  with 
structural  knowledge  tasks.  So,  the  third  study  provided  a  more  deliberate  structural  knowledge  focus 
by  using  semantic  networking  as  a  practice  and  evaluation  strategy. 

Semantic  Networking  as  an  Integrative  Strategy 

The  process  of  browsing  a  hypertext  models  the  accretion  process  in  learning  (Rumelhart  & 
Norma  l,  1978).  However,  a  significant  problem  in  learning  from  hypertext  is  the  integration  of  what 
is  acquired  while  browsing  into  the  learner's  knowledge  structure  (Jonassen,  1989b)  and  the 
restructuring  of  thos  knowledge  structures  (Rumelhart  &  Norman,  1978).  Therefore,  it  may  be 
necessary  to  provide  integration  and  restructuring  activities  to  browsing  in  order  to  ensure  learning 
from  hypertext.  Semantic  networking  software  may  be  used  as  a  cognitive  learning  strategy 
(Jonassen,  1989)  to  help  learners  integrate  information  from  hypertext.  Supplementing  browsing 
activity  with  a  semantic  networking  activity  my  provide  the  learner  with  a  tool  for  integrating 
information  into  their  knowledge  structure.  This  process  of  semantic  networking  also  elicits  the 
process  of  restructuring. 

Method 

The  third  study  was  conducted  to  assess  the  role  of  semantic  networking  as  a  cognitive 
learning  strategy  to  facilitate  structural  knowledge  acquisition.  This  study  assigned  a  group  of  the 
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students  the  responsibility  of  constructing  a  semantic  network  about  the  topic  of  the  hypertext  following 
the  study  period.  The  other  half  were  told  only  to  study  the  hypertext  to  acquire  knowledge  during  the 
study  period. 

This  third  study  included  a  second  factor,  the  type  of  structural  knowledge  support  The 
second  factor  split  both  treatment  groups  into  control  treatment  (no  structural  cues)  and  the  graphical 
browser  treatment,  as  in  the  first  study. 

Sufctfecte 

48  graduate  students  enrolled  courses  in  instructional  technology  at  a  western  college 
participated  in  this  study.  These  students  were  enrolled  in  teacher  education  and  instructional 
technology  programs. 

Instruments  and  Materials 

The  assessment  instruments  used  in  this  study  were  identical  to  those  used  in  Experiments  1 
and  2.  Two  treatments  were  developed  for  this  study  using  the  same  HyperCard  version  (Jonassen, 
Roebuck  &  Wang,  1990)  of  the  book  Hypertext  I  Hypermedia  (Jonassen,  1989)  that  was  used  in 
Experiments  1  and  2.   The  control  treatment,  consisted  of  embedded  referential  links  in  the  cards  in 
addition  to  the  "related  terms'1  nodes  attached  to  the  75  major  concepts.  Links  were  presented  in  list 
form  without  any  structural  information,  as  in  Experiments  1  and  2.  The  experimental  treatment 
provided  users  with  graphical  browser  to  use  for  navigation..  Each  graphical  browser  showed  the 
current  node  and  all  nodes  linked  directly  to  it  and  explicitly  depicted  the  nature  of  the  relationship 
between  each  of  those  nodes  (see  Figure  1  for  an  example).  Each  of  the  icons  in  the  graphical  browsers 
were  hot  buttons  which  the  learners  used  to  navigate  through  the  hypertext.  In  order  to  navigate  the 
hypertext,  users  would  click  on  any  icon  to  traverse  the  link  to  that  topic. 

The  second  experimental  factor  investigated  the  effects  of  providing  a  semantic  networking 
learning  strategy  to  the  hypertext.  In  the  experimental  treatment,  we  informed  learners  that  they 
would  be  required  to  develop  a  semantic  network  about  the  topic  of  the  hypertext  following  browsing 
and  studying  the  hypertext.  Learners  were  provided  with  with  the  semantic  networking  program, 
Learning  Tool  (Kozma,  1987).  They  had  perviously  learned  how  to  use  this  tool  (see  Figure  2  for 
example). 
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Figure  2 

Procedure 

Students  were  given  the  task  of  learning  about  an  important  new  instructional  technology, 
hypertext,  as  an  assignment  in  their  instructional  technology  course  by  using  a  hypertext,  In  a 
Macintosh  laboratory,  students  individually  interacted  with  and  studied  the  hypertext  in  order  to 
acquire  as  much  information  about  this  new  technology  as  possible.  A  monitoring  program  was 
added  to  the  stacks  to  audit  the  action  and  the  time  allotted  to  each  learner  action.  The  exeri mental 
group  was  told  that  they  would  be  responsible  for  developing  a  semantic  network,  however,  this  activity 
was  ceased  after  only  a  few  minutes.  The  treatment  was  developed  to  focus  their  attention  on 
structural  information  in  the  hypertext.  Learning  effects  were  assessed,  as  in  the  othei  studies, 
immediately  following  treatment  and  before  any  semantic  nets  were  created,  so  time-on-task  did  not 
differ  for  the  groups. 

Results,  tn  the  third  study,  learners  who  were  given  the  task  of  creating  a  semantic  network 
performed  significantly  better  on  the  relationship  judgements  task  than  the  two  groups  instructed  only 
to  study  the  materials  ,  F=7.82,  p<.01.  See  Table  3  for  means.  This  result  indicates  that  by  focusing  the 
learner's  attention  on  structural  aspects  of  the  information  in  the  hypertext,  structural  knowledge 
acquisition  improved  significantly, 
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Hypertext  Version 
Treatment      Control  Graphical  Browser 

Read  Only      4*7(1.3}  5.7  (1.8) 

Semantic  N^t  6  3  (1.7)  6.7  (1,8) 

Table  3 

Neither  the  difference  Wt-veen  the  form  of  the  hypertext  or  the  interaction  were  significant.  No 
difference*;  ;<i  recall  sco  es  '.v*,rt.  found.  However,  a  one  factor  ANOVA  comparing  scores  between  the 
four  treatment  groups  mdiuued  that  tlie  graphical  browser/semantic  networking  group  performed 
sigp^iCant;y  better  on  the  analogy  subtile,  7^=2.77,  p=.05.  It  appears  that  the  visual  support  of  the 
gn.phica<  browser  in  addition  to  for^ueiag  on  structural  relationships  enhanced  the  comparisons 
ojcess&ry  fo*  completing  analogies. 


DISCUSSION 

Cot?  )v  /  jns       ii  >c;  soi  va  firom  these  three  studies  will  be  considered  together.  Several 
issues  regtiMing  the  use  of  hypertext  for  learning  and  the  enhancement  of  structural  knowledge 
acquisition  emerged  from  these  studies. 

Attention  to  Structural  It&x -imtio a 

The  evidence  indicate;  that  to  the  extent  that  learners  attended  to  the  structural  cues  provided 
in  the  hypertexts,  they  did  acquire  some  structural  knowledge.  The  post  hoc  regression  analyses 
showed  that  students  were  in  fact  attending  to  some  of  the  structural  information.  The  number  of 
structural  cards  accessed  appeared  to  predict  structural  knowledge  acquisition  in  the  form  of 
relationship  judgements  in  two  different  forms.  The  time  spent  with  the  structural  cards  suggested  a 
relationship  between  accessing  those  cards  and  the  level  of  structural  knowledge  acquisition.  These 
weie  weak  effects,  though  with  longer  exposure  to  the  hypertext  and  the  prior  acquisition  of  hypertext 
processing  strategies,  these  findings  may  be  substantiated. 

The  most  questionable  assumption  of  our  hypotheses,  based  upon  the  results,  is  that  merely 
providing  structural  cues  in  the  user  interface  will  result  in  structural  knowledge  acquisition.  The 
fact  that  control  treatments  produced  the  highest  level  of  recall  is  consistent  with  research  that  shows 
that  without  cueing  or  practice,  learners  tend  to  recall  micropropositions  (detail)  more  readily  than 
macropropositions.  The  structural  information  presented  clearly  impeded  the  recall  of  specific  facts 
by  the  students  in  the  structural  knowledge  treatments.  However,  it  did  not  generally  result  in  greater 
structural  knowledge  acquisition.  The  conclusion  is  obvious  -  merely  showing  learners  structural 
relationships  is  probably  not  sufficient  to  result  in  structural  knowledge  encoding.  Requiring 
structural  knowledge  outcomes  in  the  form  of  thew  semantic  networking  exercise  is  able  to  produce 
enhanced  acquisition  of  structural  knowledge  though.  That  is,  getting  learners  to  focus  on  structural 
relationships  will  enhance  structural  knowledge  acquisition,  which  supports  higher  order  thinking 
in  the  form  of  analogical  reasoning. 

Cognitive  limitations  of  Browsing  Behavior 

Another  related  and  probable  reason  for  no  main  effects  in  structural  knowledge  acquisition 
is  the  "mistaken  notion  concerning  hypertext  ....that  the  arbitrary  'webs'  of  facts  in  hypertext  systems 
have  much  semantic  significance"  (Whalley,  1990,  63).  What  matters  most  in  learning  is  the 
construction  of  personally  relevant  knowledge  structures.  It  appears  that  arbitrarily  imposed 
semantic  nets  are  not  adequate  to  overcome  the  personal  ones  or  at  least  not  directly  mapped  onto  the 
learners.  It  is  becoming  increasingly  obvious  that  learning  from  hypertext  must  rely  on  externally 
imposed  or  mediated  learning  tasks  -  that  merely  browsing  through  a  knowledge  base  does  not 
engender  deep  enough  processing  to  result  in  meaningful  learning. 

According  to  Whalley  (1990),  the  most  natural  mode  of  studying  hypertext  is  browsing.  The 
question  raised  by  these  studies  is  the  extent  to  which  unconstrained  browsing  can  support  learning 
goals.  Hypertexts  are  obvious  information  retrieval  technologies.  However,  retrieval  of  information 
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is  not  sufficient  by  itself  to  result  in  meaningful  learning.  When  the  goals  of  accessing  information 
require  deeper  processing,  then  deeper  processing  is  more  likely  to  occur.  However,  simply  browsing 
hypertext  is  not  engaging  enough  to  result  in  more  meaningful  learning.  It  may  well  be  that  hypertext 
is  not  be  very  appropriate  for  highly  structured  learning  tasks,  as  Duchastel  (1990)  suggests. 

Hypertext  Processing  Strategies 

Students  in  these  studies  appeared,  as  expected,  to  lack  hypertext  processing  strategies 
(Jonassen,  1989),  which  likely  precluded  the  most  effective  use  of  the  technology.  Hypertext,  and  the 
greater  learner  control  of  instruction  that  it  entails,  is  a  novel  form  of  instruction  for  these  learners. 
The  more  novel  the  appearance  of  the  hypertext  (ie.  graphical  browser  version),  the  more  negatively 
the  students  reacted  to  it.  A  fair  evaluation  of  learning  from  hypertext  can  only  come  from  hypertext- 
literate  learners  who  have  developed  a  useful  set  of  strategies  for  navigating  and  integrating 
information  form  hypertext.  Experiments  with  learning  from  hypertext  must  assume  the 
responsibility  of  preparing  learners  to  use  this  relatively  novel  technology  before  studying  its  effects. 

Structural  Knowledge  Acquisition 

The  novel  form  of  instruction,  hypertext,  was  only  exacerbated  by  the  novel  structural  cues 
and  structural  learning  outcomes.  The  explicit  provision  of  structural  information  was  unusual  to 
these  students.  The  requirement  to  process  that  information  and  integrate  it  into  their  own  knowledge 
structures  was  even  more  novel  to  them,  which  negatively  affected  their  performance  on  the  tasks. 
The  poor  performance  of  learners  on  higher-order,  structural  knowledge  tasks  washed  out  most  of  the 
treatment  effects.  This  calls  into  question  the  ability  of  learners  to  engage  in  meaningful  learning 
rather  than  information  retrieval  from  hypertext,  especially  in  the  context  of  a  learning  assignment. 
Clearly  when  students  used  structural  knowledge  learning  strategies,  they  can  improve  their 
structural  knowldge  acquisition. 

Individual  Differences  and  Learning  from  Hypertext 

As  expected,  individual  differences  interacted  with  learning  from  hypertext.  Field 
independent  processors  generally  prefer  to  impose  their  own  structure  on  information  rather  than 
accommodate  the  structure  that  is  implicit  in  the  materials.  In  the  second  study,  field  independent 
learners  were  impeded  by  the  structural  information  provided  in  completing  the  recall  task. 
However,  on  structural  knowledge  outcomes,  they  were  the  only  learners  able  to  successfully  use  the 
structural  cues  to  acquire  more  structural  knowledge  information  than  field  dependent  learners.  It  is 
likely  that  field  independent  learners  are  better  hypertext  processors,  especially  as  the  form  of  the 
hypertext  becomes  more  referential  and  less  overtly  structured. 
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Introduction 

Collaboration  in  the  utilization  and  construction  of  hypertext  knowledge  bases  is  perhaps 
the  most  important  characteristic  of  successful  learning-oriented  hypermedia  applications  (Duffy 
&  Jonassen,  in  press;  Jonassen,  1989).  Collaboration  engages  learners  in  constructive  processing 
of  the  information  rather  the  mere  reception  and  shallow  encoding  of  information,  resulting  in 
more  meaningful  learning.  Learners  have  been  shown  U>  be  better  able  to  engage  in  rational 
argumentation  as  a  result  of  collaborative  construction  of  a  hypermedia  knowledge  base  (Landow, 
1989). 

This  report  describes  the  results  of  a  process  for  collaboratively  annotating  a  book  about 
hypertext  design  and  the  implications  of  that  process  for  designing  collaborative  hypertext 
environments.  The  oook  was  the  product  of  a  NATO-sponsored  Advanced  Research  Workshop, 
"Designing  Hypertext/Hypermedia  for  Learning"  (Jonassen  &  Mandl,  1990),  which  was  held  in 
Rottenburg  am  Neckar,  FRG,  from  July  3-8,  1989.  This  paper  describes  the  process  and  results  of  a 
collaborative  annotation  process  for  inserting  internal,  conceptual  linking  in  a  book  on 
hypermedia  design.  The  results  of  that  process,  we  believe,  have  implications  for  the  design  of 
collaborative  hypermedia  environments. 

Context 

The  idea  for  the  workshop  resulted  from  the  burgeoning  interest  in  hypertext  during  the 
latter  1980s,  combined  with  the  frustrating  lack  of  literature  on  learning  applications  for 
hypertext.  There  was  little  evidence  in  1988  that  hypertext  could  successfully  support  learning 
outcomes.  A  few  projects  were  investigating  hypertext  for  learning,  but  few  conclusions  were 
available  and  little  if  any  advice  on  how  to  design  hypertext  for  learning  applications  was 
available.  Could  hypertext  support  learning  objectives?  What  mental  processing  requirements 
are  unique  to  learning  outcomes,  and  how  do  the  processing  requirements  of  learning  outcomes 
interact  with  unique  user  processing  requirements  of  browsing  and  constructing  hypertext? 
Should  hypertext  information  bases  be  restructured  to  accommodate  learning  outcomes?  Should 
hypertext  user  interfaces  be  manipulated  in  order  to  support  the  task  functionality  of  learning 
outcomes?  Does  the  hypertext  structure  reflect  the  intellectual  requirements  of  learning  outcomes? 
What  kinds  of  learning-oriented  hypertext  systems  were  being  developed  and  what  kinds  of 
assumptions  were  these  systems  making?  These  and  other  questions  demonstrated  the  need  for 
this  workshop. 

The  workshop  took  place  over  a  five  day  period  in  a  conference  center  in  Rothenburg  am 
Neckar  in  southwestern  Germany.  It  was  hosted  by  the  Deutsches  Institut  fur  Fernstudien  at  the 
University  of  Tubingen.  The  workshop  included  a  number  of  presentations  by  various 
researchers  focusing  on  several  themes,  including: 


Hypermedia  and  Learning 

David  Jonassen  Problems  and  Issues  in  Designing 

Scott  Grabinger  Hypertext/Hypermedia  for  Learning 

John  Leggett  Hypertext  for  Learning 

George  Landow  Popular  Fallacies  About  Hypertext 
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Peter  Whalley 

Rainer  Hammwohner 
Ray  McAleese 

Piet  Kommers 


Jakob  Nielsen 
Patricia  Wright 
Andrew  Dillon 


Thomas  Duffy 

Terry  Mayes 

Armando  Oliveira 
Duarte  Costa  Perreira 

Wil  Verrick, 
Anja  Lkoundi 

Cliff  McKnight 
John  Richardson 
Andrew  Dillon 

Otmar  Folsche 


Martin  Richart 
Tom  Rudebusch 


Alex  Romiszowski 


Gary  Marchionini 


Models  of  Hypertext  Structure  and  Learning 

Designing  the  Information  Model 

Macro-Operations  for  Hypertext  Construction 

Concepts  as  Hypertext  Nodes:  The  Ability  to  Learn 
While  Navigating  Through  Hypertext  Nets 

Graph  Computation  as  an  Orientation  Device  in 
Extended  and  Cyclic  Hypertext  Networks 

Designing  the  Hasi  Interface 

Evaluating  Hypertext  Usability 

Hypertexts  as  an  Interface  for  Learners:  Some  Human  Factors 

Dsigning  the  Human-Computer  Interface  to 
Hypermedia  Applications 

Hypermedia  and  Instruction 

Hypermedia  and  Instruction:  Where  is  the  Match? 

Learning  About  Learning  from  Hypertext 

Psych opedagogic  Aspects  of  Hypermedia 
Courseware 

From  Instructional  Text  to  Instructional 
Hypertext:  An  Experiment 

Journal  Articles  as  Learning  Resource:  What  Can 
Hypertext  Offer? 


Hypertcxt/Hypermedia-like  Environments  and 
Language  Learning 

Collaboration  in  Hypermedia  Environments 


Hypermedia  Design  Process 

The  Hypertext/Hypermedia  Solution  But 
What  Exactly  is  the  Problem? 

Evaluating  Hypermedia-Based  Learning 


Conceptual  Foundations  for  Designing  Hypermedia  Systems  fci 


Eric  Bruillard 
Gerard  Weidenfeld 


Some  Examples  of  Hypertext's  Applications 


Max  Muhlh&user 


Hyperinformation  Requirements  for  an  Integrated 
Authoring/Learning  Environment 


Norbert  Streitz 
J6rg  Hannemann 

Daniel  Russell 


Elaborating  Arguments:  Writing,  Learning,  and 
in  a  Hypertext  Based  Environment  for  Authoring 

Alexandria:  A  Learning  Reseources  Management 
Architecture 


Problem 

The  conference  consisted  of  the  previous  presentations,  hardware  demonstrations,  sharing 
and  browsing  of  hypertexts,  and  extensive  discussions  about  all  of  the  above.  These  latter 
experiences  cannot  be  effectively  shared  with  the  reader  of  the  proceedings.  The  topic  of  the 
conference,  hypermedia;  represents  a  dynamic  knowledge  representation  form.  It's  most 
dynamic  capacity,  represented  in  the  presentation  by  Richartz  and  Riidebusch,  is  the  collaborative 
contribution  of  different  individuals  to  the  knowledge  base.  In  collaborative  learning 
environments,  the  individuals  involved  are  participants  rather  than  mere  respondents. 
Therefore,  we  attempted  to  share  the  experiences  of  the  conference,  especially  the  dialogue,  with  the 
readers  of  the  book  while  representing  the  collaborative  nature  of  the  conference  experience. 
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Annotations 


Page  Layout  of  Book 


Figure  1 
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Method 

Procedure 

Following  the  workshop,  we  asked  each  of  the  contributors  to  revise  their  papers  to  reflect 
the  ideas  and  discussion  shared  at  the  workshop.  Each  paper  needed  to  make  heavier  use  of 
headings  and  to  include  a  list  of  keywords.  Each  paper  needed  to  state  its  assumptions  about 
learning,  characteristics  of  hypertext  (one  of  the  conclusions  of  the  workshop  was  that  we  should 
use  the  more  generic  term,  hypermedia,  rather  than  hypertext),  and  provide  some  advice  or 
guidelines  for  designing  hypermedia  for  learning.  These  papers  were  collected  and  edited.  They 
were  re-organized  into  sections  that  reflected  some  of  the  themes  of  the  workshop. 

After  receiving  all  of  the  edited  contributions,  the  text  was  initially  set  using  a  desktop 
publishing  system.  The  text  of  each  paper  was  was  set  in  the  larger  inside  column  of  each  page, 
leaving  the  outside  column  blank. 

A  copy  of  the  entire  book  without  any  annotations  was  then  sent  to  each  participant  in  the 
workshop.  Each  participant  was  asked  to  read  and  annotate  the  other  papers.  Participants  were 
given  the  option  of  using  three  types  of  annotations.  The  primary  type  was  in  the  form  of  qualified 
annotations  (McAleese,  Anderson  &  Duncan,  1982;  McAleese  &  Duncan,  1983).  Qualified 
annotations  consisted  of  predefined  relations  to  be  used  to  connect  the  text  adjacent  to  the 
annotation  with  other  text  in  the  book.  This  type  of  annotation  is  similar  to  using  a  constrained  set 
of  asymmetric  (two-way)  referential  links  in  hypertext.    The  qualified  annotations  used  to  link 
ideas  in  the  book  included: 

example  -  specific  instances  of  current  topic 

recommendation  -  principle  for  designing  hypermedia 

background  -  conceptual/theoretical  foundations 

explanation  -  expands  or  elaborates  on  current  topic 

corroboration  -  supports  current  topic 

illustration  -  visual  illustration  of  topic 

contrast  -  contradictory  interpretation  or  description 

definition  -  definition  of  idea  or  topic 

results  -  relevant  research  results 

parallel  -  similar  interpretation  or  description 

implication  -  inferential  conclusion 

methodology  -  description  of  relevant  research  or  design  methodology 
By  following  these  annotated  links,  the  reader  may  gain  additional  information  (for  example, 
explanation)  about  the  topic  ihat  is  defined  by  the  link.  So,  an  annotatv  ?  such  as  "DJ  explanation 
4,13"  means  that  DJ  (a  list  of  partk  \  ants  and  their  initials  was  provided  in  the  introduction  to  the 
book)  claims  that  there  is  an  explanation  of  the  idea(s)  adjacent  to  the  annotation  that  can  be  found 
on  page  4.13.  To  support  this  process,  it  was  necessary  to  provide  two  forms  of  pagination  in  the 
book,  a  running,  sequential  pagin^uAwi  :  end  a  chapter  and  page  listing  (eg.  4,13),  The  fomer  was  a 
required  by  the  publisher,  and  the  latter  form  was  used  to  support  the  linking. 

The  second  type  of  annotations  were  unqualified.  Participants  could  provide  short 
comments  about  the  text  adjacent  to  the  comment.  These  were  to  be  used  when  none  of  the  qualified 
annotations  applied  or  when  there  was  no  other  text  in  the  book  to  which  to  refer  the  reader. 

The  third  type  of  annotation  included  references  to  external  documents  that  the  author  of 
the  text  being  annotated  did  not  include  in  the  paper.  These  document  references  were  to  be  added 
when  the  external  document  has  relevance  to  the  discussion.  These  would  enable  the  reader  to 
follow  up  on  particular  discussions  more  readily.  These  are  similar  to  symmetric  (one  way 
links)  in  hypertext, 

These  annotations  were  received  by  the  primary  editor  via  electronic  mail,  The  electronic 
mail  conference  account  automatically  reflected  each  submission  of  annotations  to  all  of  the 
workshop  participants  via  several  electronic  mail  networks.  This  provided  each  author  with  the 
opportunity  to  read  how  other  workshop  participants  reacted  to  his/her  chapter.  Each  author  then 
had  the  opportunity  to  amend  their  chapter  in  order  to  accommodate  or  react  to  the  pertinent 
annotations  and  comments.  Limited  changes  to  the  document  were  made  to  the  text,  so  long  as  it 
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did  not  affect  pagination.  Finally,  the  annotations,  comments,  and  references  were  added  to  the 
outside  column  of  the  book  (see  Figure  1),  which  was  then  printed  out  in  its  final  form. 

In  addition  to  the  annotations,  other  common  hypermedia  access  tools,  such  as  contents 
lists  and  keywords  for  each  chapter,  are  provided. 

Following  the  annotation  process,  a  survey  was  sent  to  each  of  the  participants  to  ascertain 
what  the  advantages  and  difficulties  of  the  process  were  if  they  submitted  annotations.  We  were 
also  attempting  to  ascertain  how  the  process  could  be  facilitated  and  what  benefits  accrued. 

Rationale 

This  elaborate  and  iterative  process  was  undertaken  for  a  number  of  reasons.  First,  we 
hoped  that  the  quality  of  the  ideas  and  writing  would  improve  if  subjected  to  the  scrutiny  of  peers. 
Second,  we  wanted  the  book  to  exhibit  some  of  the  characteristics  of  hypermedia.  The  annotations 
represent  internal  links.  The  references  represent  external  links  to  other  documents.  Were  this 
a  true  hypermedium,  the  references  would  consist  of  links  that  transport  the  reader  directly  to  the 
relevant  portion  of  the  referenced  document.  That  is  not  possible  in  print  without  aggregating  all 
of  the  relevant  references  in  one  book.  Page  and  copyright  restrictions  preclude  that.  Third,  we 
wanted  the  book  to  reflect  some  of  the  themes,  issues,  and  opinions  discussed  but  not  resolved  at  the 
workshop  —  to  engage  the  reader  in  some  of  the  dialogues  that  engaged  the  participants  during  the 
workshop.  Finally,  we  wanted  to  continue  the  dialogue  begun  at  he  workshop  via  an  electronic 
mail  conference.  This  represented  an  opportunity  to  apply  an  important  technology  to  facilitating 
a  collaborative  process. 

Results  and  Discussion 

Over  300  annotations  were  entered  on  the  network.  These  were  received  from  only  65%  of 
the  workshop  participants.  The  overwhelming  majority  of  the  annotations  were  qualified. 

The  two  most  common  types  of  annotation  were  parallel  and  contrast,  indicating  that 
participants  most  often  agreed  and  disagreed.  Overall  though,  combining  the  parallel 
annotations  with  the  three  next  most  common  annotations  (corroboration,  example,  and 
explanation)  yields  more  than  two  thirds  of  the  annotations.  All  of  these  annotations  are 
supportive  of  each  other,  either  agreeing  with  the  ideas  being  presented,  helping  to  explain  them,  or 
providing  additonal  examples  which  is  also  explanatory.  So  rather  than  using  annotation  as  an 
opportunity  to  contrast  the  diverse  opinions  presented  at  the  workshop,  participants  chose  to  publicly 
agree  with  each  other. 

There  was  considerable  unevenness  in  the  number  of  annotations  provided  from  various 
participants.  A  number  of  participants  refused  to  participate  in  the  process  on  principle.  Their 
claim  was  that  the  book  could  not  function  as  hypertext  and  that  to  force  such  a  structure  onto  the 
book  destroyed  the  continuity  and  coherence  of  the  book.  The  most  frequent  comment  receieved  via 
the  survey  or  from  informal  comments  contributed  over  the  e-mail  network  was  that  the  process 
was  considerably  more  ditfij  lit  and  time  consuming  than  anticipated  at  the  conference.  The 
annotation  process  was  perceived  by  those  who  did  so  as  being  much  more  arduous  than  they  had 
anticipated.  As  one  participant  commented,  "Linking  is  easier  said  than  done."  The  primary 
difficulty  was  the  memory  load  required  to  holding  the  contents  and  approximate  location  of  the 
contents  of  a  300  page  book  in  memory  and  relating  it  all  to  the  rest  of  the  content.  Some 
participants  evolved  linking  strategies  for  reducing  memory  load.  They  read  the  book  mainly  to 
place  links  from  other  articles  to  their  own  article  and  vice  versa.  The  advantages  of  the  linking 
process  related  to  a  focusing  of  the  issues  and  concerns. 
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Table  1 
Summary  of  Annotations 

Most  of  the  annotations  were  submitted  in  blocks,  as  authors  saved  their  annotations  until 
they  could  send  them  altogether.  With  only  two  exceptions,  participants  sent  only  one  block  of 
annotations. 

There  was  also  unevenness  in  the  number  of  annotations  per  chapter.  The  early  chapters 
tended  to  be  annotated  more  heavily. 

Different  participants  used  the  linking  process  to  support  different  types  of  arguments. 
That  is,  there  were  significant  differences  in  the  types  of  links  generated  by  the  different 
participants.  Some  generated  primarily  "contrast"  links  while  others  generated  "example"  and 
explanation"  links.  While  we  had  anticipated  that  the  process  would  provide  more  of  a  venue  for 
argumentation,  it  actually  became  a  medium  for  mutual  support. 

Another  expressed  concern  was  over  the  symmetry  of  the  links.  The  process  provided  only 
for  one-way  links  with  no  return  provided.  Some  particpants  expressed  concern  that  such  links 
were  not  as  meanigful  as  assymmatric  links  that  go  both  ways. 

A  number  of  participants  commented  on  the  static  nature  of  the  knowledge  base.  Concern 
was  expressed  about  the  difficulties  of  annotating  a  static  knowledge  base  (a  print  book)  and  how 
much  more  difficult  that  might  become  in  a  dynamic,  collaboratively  constructed  hypermedia 
knowledge  base. 

Finally,  the  syntactic  requirements  of  sharing  annotations  in  an  electronic  mail 
environment  constrained  the  sharing  process. 

Implications  iir  Hypertext 
From  the  annotations  and  comments  that  accompanied  them,  the  following 
recommendations  can  b  made. 


45f; 

6 


4rv  •** 


•  Since  each  author  is  most  familiar  with  his/her  own  chapter,  then  annotate  the  other  chapters  in 
the  book  by  providing  annotations  back  to  your  own  chapter  or  read  the  text  with  the  thought  of 
annotating  your  own  contribution  by  relating  it  to  the  others. 

•  Depending  upon  the  purpose  of  the  collaborative  hypertext,  develop  a  tighter  argument  structure 
using  fewer  annotation  types.  Collaborative  hypertext  systems,  such  as  IBIS  (Conklin  & 
Begemen,  1987),  provide  a  limited  set  of  options.  Users  can  identifiy  issues,  add  positions  to 
those  issues,  and  then  add  arguments  to  the  positions.  We  use  such  a  structure  to  organize 
discussion  in  a  doctoral  seminar.  When  individual  wnat  to  add  an  issue,  they  are  given  the 
choice  of  declaring  advantage,  disadvantage,  application,  problem  or  limitation  nodes  when 
comparing  models  of  instruction. 

•  Provide  multiple  access  points  to  the  hypertext  being  annotated.  Linear  documents  are  more 
closely  scrutnized  in  the  beginning  with  interest  and  commitment  waning  toward  the  end  of  the 
document.  Help  users  to  access  the  document,  taat  is  begin  processing  the  document,  in  different 
places  to  ensure  even  annotations. 

•  Develop  some  form  of  cueing  help  to  asist  the  annotation  process.  A  bookmarking  type  of  cues 
that  can  be  placed  in  the  hypertext  at  different  locations  would  reduce  the  memory  load. 

•  Provide  the  option  of  creating  symmetric  or  assymmetric  links.  If  two  way  links  are  used,  ten 
require  that  participants  define  the  nature  of  the  link  in  both  directions,  since  it  is  bound  to 
differ. 
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Introduction 


In  the  current  information  age,  society  is 
continually  being  influenced  and  reshaped  by  rapid  and 
pervasive  changes  in  technology.     Education  has  not 
been  immune  to  these  changes,  although  it  has  been  slow 
to  adapt  (Braun,   1990,  Dalton,   1989)  .     In  particular, 
the  advent  of  the  low  priced  and  accessible 
microcomputer  has  placed  the  power  and  flexibility  of 
computing  into  the  hands  of  the  individual.    A  major 
challenge  for  instructional  designers  is  to  learn  how 
to  make  effective  use  of  its  capabilities  to  assist 
people  to  learn. 

An  exciting  extension  of  the  microcomputer 
revolution  is  the  interactive  computer-based  learning 
system  most  commonly  referred  to  as  Computer-Based 
Interactive  Video  (CBIV) ,  a  hypermedia  system  which 
combines  the  power  of  the  computer  to  support  student 
interaction  with  the  richness  of  the  various  audio  and 
visual  media.     The  main  benefit  of  hypermedia  (Kinsie  & 
Berdel,  1990)   is  to  facilitate  learning  by  "making 
networks  of  related  information  available  in  a  content- 
appropriate  multimedia  format  (p. 62)." 

According  to  Jonassen  (1984) ,  the  major  advantage 
of  CBIV  is  that  it  can  be  both  adaptive  and 
interactive.    He  defines  adaptive  as  "the  ability  to 
adapt  or  adjust  the  presentation  sequence,  mode  or  sign 
type  to  meet  a  variety  of  instructional 
requirements..."  and  interactive  to  mean  "that  the 
program  engages  the  learner  to  participate  in  a  variety 
of  ways  that  utilize  learner  responses..."   (p. 21). 

Hannafin  (1989)  views  interactivity  as  providing 
one  or  more  instructional  functions  from  simple 
procedural  control  to  causing  differentiated  levels  of 
cognitive  processing.    These  functions  of  interaction 
include  confirmation  of  response,  learner  control  of 
pacing  and  lesson  sequencing,  inquiry  (glossaries  and 
libraries) ,  navigation,  and  elaboration,  techniques 
which  allow  learners  to  combine  known  with  to-be- 
learned  lesson  information. 

Learner  Disorientation 

CBIV,  while  providing  such  flexibility,  is  nob 
without  problems.     One  is  the  potential  for  learner 
disorientation,  the  loss  of  one's  sense  of  location  or 
of  the  structure  of  the  material.     Navigation  is  the 
most  commonly  identified  user  problem  in  hypermedia 
(Jonassen,  1989,  Kinzie  &  Berdel,  1990,  Rezabek  & 
Ragan,   1989) .     Learners  can  easily  become  lost  and 
frustrated  and  may  give  up  without  acquiring  -my 
information  from  the  program. 
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Another  problem  is  cognitive  overload.  Jonassen 
(1989)  also  notes  that  the  exponentially  greater  number 
of  learning  options  available  to  learners  places 
increased  cognitive  demands  upon  learners  that  they  are 
often  unable  to  fulfil.     Tripp  and  Roby  (1990)  claim 
that  disorientation  leads  to  the  expenditure  of  more 
mental  effort  to  maintain  a  sense  of  orientation  in  the 
program  which  in  turn  reduces  the  mental  resources 
available  for  learning. 

Instructional  Design  for  Hypermedia 

How  might  one  then  allay  such  disadvantages  of 
hypermedia  while  capitalizing  on  its  advantages?    In  a 
flexible  environment  such  as  hypermedia,  self- 
regulation  -  skills  for  self-directed  learning  -  is 
essential   (Kinzie  &  Berdel,  1990).     Systems  can  be 
designed  to  make  the  exercise  of  learner  control  and 
self-regulation  successful  and  to  enhance  a  learner's 
continuing  motivation.     Kinzie  &  Berd/  .    (1990)  suggest 
the  use  of  such  techniques  as  "motivating  overviews" 
[original  emphasis]  that  encourage  users  to  explore 
different  parts  of  the  program  and  system  maps  to 
locate  current  position  and  to  move  to  a  desired 
location. 

While  Kinzie  &  Berdel  (1990)  emphasize  self- 
regulation  and  motivation,  Jona^seri  (1989)  focuses  on 
cognitive  structure.     In  his  view,  effective  browsers 
of  hypermedia  must  be  able  to  monitor  their  own 
comprehension  of  the  information  presented  and  to 
select  appropriate  strategies  that  facilitate 
integrating  and  synthesizing  information.     If  the 
material  can  be  organized  to  replicate  content  or 
expert  knowledge  structures,  then  it  would  better 
provide  the  needed  anchors  for  re-structuring  the 
learner's  knowledge. 

Jonassen  (1989)  suggests  that  designers  of 
hypermedia  should  be  concerned  with  two  aspects:  the 
information  model  p~  rented  by  the  program  and  the  user 
interface.     The  foin.er  describes  how  to  organize  or 
structure  the  information.     The  user  interface 
describes  the  tools  that  the  user  has  for  accessing  and 
navigating  through  the  system.     These  include  indexes 
and  menus,  cross-referencing  and  hypermaps.     A  similar 
notion  to  system  maps,  hypermaps  provide  a  graphical 
view  of  the  program  structure,  allowing  the  user  to 
select  a  node  on  the  hypermap  and  be  taken  immediately 
to   -hat  part  of  the  program.     The  theoretical  advantage 
of  i sing  a  hypermap  (Jonassen,  1989)   is  that  it  should 
enhrnce  the  learner's  structural  knowledge  (the 
knowledge  of  interrelationships  between  ideas)  of  the. 
information  in  the  program. 
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Instructional  Organizers 


One  approach  to  enhance  structural  knowledge  is 
the  use  of  instructional  organizers.    Tripp  &  Roby 
(1990)  suggest  a  traditional  strategy,  the  advance 
organizer  (Ausubel,  1960,  1963).     Other  related 
organizational  approaches  include  the  structured 
overview  graphic  organizer  (Barron,  1969)  and  the 
pictorial  graphic  organizer  (Hawk,  McLeod  &  Jonassen, 
1985) ,  both  derivatives  of  the  advance  organizer.  The 
pictorial  graphic  organizer  can  be  either  participatory 
(providing  spaces  to  be  filled  in  by  the  learner)  or 
final  form  (spaces  already  completed  when  given  to  the 
learner) .     Both  structured  overviews  and  pictorial 
graphic  organizers  resemble  the  hypermap  and  could 
potentially  be  used  to  assist  navigation  in  a  CBIV 
program.     All  three  organizers  are  designed  to  relate 
the  material-to-be-learned  to  the  learner's  existing 
cognitive  structure  to  facilitate  learning  and 
retention. 

Extensive  research  on  the  advance  organizer  has 
produced  a  conflicting  evidence  concerning  its 
effectiveness  in  facilitating  learning.  Several 
reviews  have  been  critical  (Barnes  &  Clawson,  1975, 
Hartley  &  Davies,   197  6,  Macdonald-Ross,   1978,  McEneany, 
1990) ,  while  others  have  been  supportive  (Lawton  & 
Wanska,  1977,  Mayer,   1979a,   1979b i  Luiten,  Ames  & 
Ackerson,  1980,  Stone,  1984).     Overall,  advance 
organizers  appear  to  have  a  modest  effect  upon  learning 
and  retention  (Kenny,   1990) .     While  noting  far  fewer 
research  results,     Kenny  also  found  a  modest  effect 
upon  learning  and  retention  for  structured  overview  and 
pictorial  graphic  organizers. 

The  few  studies  of  organizers  with  CBIV-based 
instruction  have  mostly  focused  on  the  advance 
organizer.     Carnes,  Lindbeck  &  Griffin  (1987)  reported 
no  significant  differences  for  advance  organizers  with 
a  computer-based  physics  tutorial.     Krahn  &  Blanchaer 
(1986)   found  a  statistically  significant  difference  in 
favor  of  advance  organizers  in  a  computer-based  medical 
simulation  but  assessed  learning  with  a  very  small 
immediate  recall  test  (six  questions) .     Theory  holds 
that  an  advance  organizer  is  unlikely  to  assist 
immediate  recall  and  should  be  expected  to  have  its 
strongest,  effect  on  retention  (Mayer,  1979a,   1979b) . 
Research  results  have  tended  to  support  this  assertion 
(Luiten,  Amis  &  Ackerson,  1980,  Stone,   1984,  Kenny, 
1990)  . 

Tripp  &  Roby  (1990)  report  the  use  of  an  advance 
organizer  with  a  Japanese-English  hypertext-based 
lexicon.     Half  the  group  using  the  advance  organizer 
also  received  a  visual  metaphor.     They  speculated  that 
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a  statistically  significant  negative  effect  for  the 
interaction  between  the  two  techniques  may  have  negated 
any  effect  for  the  advance  organizer. 

Kenny,  Grabowski,  Middlemiss  &  Van-Neste-Kenny 
(1991)  compared  participatory  pictorial  graphic 
organizers  to  identical  final  form  organizers  when  used 
with  a  commercial  CBIV  program  on  nursing  elderly 
patients  with  pulmonary  disease.     The  participatory 
graphic  organizer  group  outperformed  the  final  form 
group  on  a  test  of  immediate  recall.     The  difference 
was  large  (1.65  points  on  an  18-item  multiple  choice 
test)  but  was  not  statistically  significant.  However, 
the  final  form  group  obtained  a  slightly  higher  (0.18 
points)  mean  score  on  the  retention  test  (given  after 
one  week) .     As  well,  surprisingly,  the  final  form  group 
made  a  considerable  gain  (1.77  points)   from  the 
immediate  recall  test  to  the  retention  test.     One  would 
normally  expect  some  forgetting  to  occur.  This 
difference,  however,  was  not  statistically  significant. 
Regardless,  the  result  is  consistent  with  advance 
organizer  theory  (Mayer,  1979a,   1979b)  and  may  indicate 
that  the  final  form  graphic  organizer  affects  learning 
in  the  same  manner  as  an  advance  organizer. 


Generative  Learning 

Pictorial  and  structured  overview  graphic 
organizers  both  appear  to  be  most  effective  when 
presented  in  participatory  form,  that  is,  to  be  filled 
in  or  constructed  by  the  learner  (Kenny,  1990).  Thus, 
Wittrock' s  (1974)  generative  learning  hypothesis  may 
provide  a  stronger  basis  for    xplaining  the 
effectiveness  of  these  techniques  than  does  advance 
organizer  theory.     Wittrock  claims  that  "it  is  the 
learner's  interpretation  of  and  processing  of  the 
stimuli,  not  their  [the  stimuli]  nominal 
characteristics,  which  is  primary"  (p. 88).     In  more 
simple  terms,  learners  must  construct  their  own  meaning 
from  teach- ng  (Wittrock,  1985) .     Doctorow,  Wittrock  & 
Marks  (1978)   indicate  that  this  involves  not  only 
generating  meaning  but  overt  activities  as  well;  such 
as,  generating  associations  among  words,  generating 
pictures,  etc.     By  way  of  example,  Wittrock  (1990, 
p.  347)  expresses  the  view  that  even  reading  "involves 
generative  cognitive  processes  that  create  meaning  by 
building  relations  (a)  among  parts  of  the  text  and  (h) 
between  the  text  and  what  wa  know,  believe  and 
experience."    That  is,  wher  we  read,  we  generate 
meaning  by  relating  parts  of  the  text  to  one  another 
and  to  our  memories  and  knowledge. 
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Hypotheses 


This  study  was  intended  to  compare  the  use  of 
three  instructional  organizers  -  the  advance  organizer 
(AO) ,  the  participatory  pictorial  graphic  organizer 
(PGO)  and  the  final  form  pictorial  graphic  organizer 
(FGO)  -  in  the  design  and  use  of  CBIV  programs.  That 
is,  it  attempted  to  answer  the  question  of  which  form 
of  instructional  organizer  (less  generative  or  more 
generative)  would  most  effectively  facilitate  learning 
and  retention  when  used  with  this  form  of  hypermedia. 
It  was  hypothesized  that  (1)  a  PGO  would  be  more 
effective  in  facilitating  immediate  recall  than  would 
either  an  AO  or  the  equivalent  FGO,    (2)  a  PGO  would  be 
more  effective  in  facilitating  retention  than  would  the 
othf.r  two  techniques  and  (3)  an  FGO  and  an  AO  would  not 
substantially  differ  from  each  other  in  their  effect  on 
r  mediate  recall  or  retention.    A  PGO  requires  the 
learner  to  fill  in  various  blanks  on  the  organizer  and 
was  considered  to  be  a  more  generative  technique  than 
either  its  FGO  version  or  an  AO.     To  complete  a  PGO, 
the  learner  must  select  information  from  the  CBIV 
program  and  write  it  in  the  appropriate  place  on  the 
organizer.     When  using  an  FGO,  the  lf.arner  is  asked 
only  to  read  and  remember  the  information  presented  by 
the  organizer.     Similarly,  the  AO  presents  information 
at  a  higher  or  more  abstract  level  and  is  also  meant  to 
be  read  and  remembered.     Thus,  it  was  expected  that  the 
PGO  would  elicit  a  generative  activity  from  the  learner 
while  the  FGO  and  AO  would  depend  on  their  nominal 
characteristics  for  any  effect. 


Methods 

Subjects 

The  study  sample  consisted  of  sixty-two 
university  nursing  students  and  faculty  from  a 
university  College  of  Nursing  in  the  northeastern 
United  States.     All  subjects  were  female.     This  sample 
included  three  blocks  or  strata:     third  year  (Juniors) 
undergraduates  (n  =  16) ,  fourth  year  (Seniors) 
undergraduates  (n  =  19)  and  graduates  (n  =  27) .  The 
graduate  block  was  comprised  of  twenty-one  Master's 
degree  students  and  six  faculty  members.     The  posttest 
data  for  one  graduate,  who  was  given  the  participatory 
graphic  organizer  treatment,  was  discarded  because  the 
subject  had  disregarded  treatment  instructions  and  took 
copious  notes  on  both  the  front  and  back  of  the  form. 
This  left  a  total  sample  of  sixty-one  individuals. 
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Materials 


An  American  Journal  of  Nursing  CBIV  program,  The 
Nursing  Care  of  Elderly  Patients  with  Acute  Cardiac 
Disorders  was  used  in  this  study.    This  program 
presents  two  case  studies  each  of  which  follow  the 
progress  of  an  elderly  male  patient  from  admission  to 
hospital  to  discharge.    One  patient  is  admitted  with  a 
heart  attack  while  the  other  is  treated  for  congestive 
heart  failure.    The  content  of  the  program  consists  of 
specialized  knowledge  and  skills  in  acute  cardiac  care 
such  as  the  recognition  of  age-related  changes  in 
cardiac  function,  cardiac  pathophysiology,  knowledge  of 
cardiac  drugs  and  the  reading  of  electrocardiogram 
(EKG)  rhythms.     The  case  studies  consist  of  several 
video  sequences  of  patient  -  nurse  interactions, 
including  decisions  about  how  to  treat  the  patient's 
condition  and  nursing  responses  to  an  emergency 
situation  resulting  from  treatment. 

The  program  uses  a  structured  discovery  approach 
in  which  the  learner  is  presented  with  a  video  sequence 
followed  by  a  decision  point.    At  each  point,  the 
learner  must  make  appropriate  nursing  assessment  or 
intervention  decisions  in  order  to  proceed.  These 
decisions  are  rated  on  a  scale  ranging  from  +3  to  -3. 
Those  judged  by  the  program  harmful  to  the  patient  are 
scored  -2  or  -3  and  result  in  immediate  feedback 
accompanied  by  a  request  to  try  again.     Appropriate  or 
neutral  decisions  are  accepted  without  immediate 
feedback.     The  learner  cannot  advance  until  sufficient 
appropriate  decisions  have  been  made.     The  prior  video 
sequence  is  then  replayed  and  accompanied  by  voice-over 
analysis  by  an  experienced  nurse. 

Independent  variables 

The  primary  independent  variables  were  the  three 
instructional  treatments:  the  PGO  (considered  the  most 
generative),  the  FGO  and  an  expository  AO.     The  three 
organizers  used  in  the  study  were  developed  by  two 
subject  matter  experts  (on  adult  medical-surgical  and 
critical  care  nursing)  and  the  author,  whose  expertise 
is  in  instructional  design. 

The  graphic  organizers  were  constructed  according 
to  Hawk,  McLeod  and  Jonassen  (1985) .     They  were 
identical  except  that  the  FGO  was  completed  while 
various  blanks  were  left  to  complete  in  the  PGO.  They 
were  designed  to  provide  a  jombined  chronological  and 
comparison  -  contrast  structure,  that  is,  they  followed 
the  admission-to-hospital  through  treatment  and 
discharge  sequence  of  the  program  and  provided  side-by- 
side  comparison  of  the  stages  for  each  case  study. 
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The  AO  was  constructed  in  expository  form 
according  to  the  prescriptions  of  Ausubel  (1963,  1978) 
and  Mayer  (1979a).     The  content  of  the  AO  consisted  of 
a  discussion  of  clinical  decision-making  and  the 
nursing  process.     This  included  an  overview  and  a  brief 
discussion  of  the  five  steps  of  the  nursing  process: 
assessment,  diagnosis,  planning,  intervention  and 
evaluation.     The  organizer  also  related  the  nursing 
process  to  collaborative  problems  which  are  solved  in 
conjunction  with  medical  personnel.     All  subjects  would 
have  studied  this  process  in  past  and  had  varying 
degrees  of  familiarity  with  the  material. 

Dependent  Variables 

There  were  two  dependent  variables  in  this  study: 
immediate  recall  and  retention.     Immediate  recall  was 
operational ized  as  a  test  of  learning  from  the  CBIV 
program  given  immediately  after  completion  of  the 
program  and  retention  as  a  similar  measure  of  learning 
given  one  week  later.     Both  were  measured  by  two, 
equivalent  form,  24-question,  multiple  choice, 
posttests  developed  by  the  producers  of  the  CBIV 
program.     The  posttests  were  criterion-referenced  and 
nearly  identical  in  format  and  question  type.     In  most 
cases  they  differed  only  in  terms  of  situational 
information  such  as  patient  assessment  data.     The  test 
questions  were  developed  on  the  basis  of  program 
objectives  and  follow  nursing  care  scenarios 
resembling,  but  not  identical  to,  the  CBIV  program 
itself  (Rizzolo,  personal  communication) .  All 
questions  tested  nursing  assessments  and  interventions 
for  which  the  program  was  designed  to  provide  practice. 

Internal  consistency  reliability  (Kuder- 
Richardson  Formula  21)  was  0.66  for  Form  A  and  0.64  for 
Form  B.     The  stability  and  equivalence  reliability 
estimate  was  0.59.     Popham  (1981)   indicates  that  this 
yields  the  lowest  estimate  of  test  reliability  since 
two  factors  -  time  delay  and  differences  between  the 
test  forms  -  operate  to  reduce  consistency  of 
measurement.     Estimates  of  0.70  and  lower  may  be 
considered  acceptable  for  equivalence  and  stability. 


Procedures 


Subjects  were  randomly  assigned  to  the  three 
treatment  groups  -  PGO,  FGO  or  AO  -  by  block  (strata). 
All  subjects  were  asked  to  read  their  organizer 
immediately  prior  to  doing  the  CBIV  program.  The 
purpose  of  the  organizer  was  explained  at  that  time. 
Those  in  the  PGO  treatment  group  were  also  asked  to 
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fill  in  the  blanks  in  the  organizer  either  during  or 
immediately  after  the  CBIV  program.    The  subjects  were 
given  complete  freedom  to  proceed  through  the  program 
as  they  wished  although  they  were  requested  to  attempt 
both  case  studies.     They  were,  however,  asked  not  to 
take  notes  except  where  required  by  the  PGO  treatment. 
This  was  done  to  counteract  note-taking  as  a  possible 
confounding  factor,  a  problem  which  occurred  in  the 
previous  study  (Kenny,  Grabowski,  Middlemiss  &  Van- 
Neste-Kenny,  1991) . 

Students  completed  the  first  posttest  (immediate 
recall)  immediately  after  finishing  the  CBIV  program 
and  the  second  (retention)  posttest  one  week  later. 
They  did  not  have  access  to  the  program  in  between 
testings . 


Experimental  Design 

This  study  used  a  three  by  three  by  two  factorial 
design  with  one  repeated  measure,  or  a  split  plot 
factorial  (SPF)  design  (Kirk,  1982).     The  design 
factors  for  this  study  were  the  three  treatment  groups, 
the  three  blocks  (Junior,  Senior,  Graduate) ,  and  the 
order  of  presentation  of  form  of  posttest.  All 
subjects  were  randomly  assigned  to  either  Form  A  or 
Form  B  of  the  posttest  for  the  test  of  immediate  recall 
and  given  the  other  form  for  the  retention  test.  This 
counterbalancing  was  carried  out  in  order  to  compensate 
for  any  bias  which  might  have  resulted  from  differences 
in  the  two  versions  of  the  test.     The  criterion- 
referenced  posttests  represented  the  repeated  measures 
factor. 

The  hypotheses  were  analyzed  using  planned 
contrasts.  This  was  accomplished  in  two  steps.  First, 
a  3x3x2  MANOVA  with  one  repeated  measure  was  conducted 
to  provide  a  pooled  Mean  Square  Error  term  for  the 
calculation  of  the  planned  contrasts  and  to  provide  a 
test  of  any  main  effects  for  the  blocking  and 
counterbalancing  factors  and  their  interactions  with 
the  main  treatment  effect. 

Second,  examination  of  the  hypotheses  reveals 
that  a  test  of  all  pairwise  comparisons  was  required. 
Consequently,  the  use  of  Tukey  HSD  contrasts  was 
appropriate  and,  in  this  situation,  more  sensitive  than 
the  use  of  Dunn's  procedure.     Therefore,  all  six 
hypotheses  were  tested  using  the  Tukey  HSD  procedure  at 
<*  <  0.10  level. 

Convention  holds  that  alpha  is  generally  set  at  < 
0.05  and  the  selection  of  the  0.10  level  may  be 
considered  liberal.     Cohen  (1990),  however,  labels  the 
former  a  "convenient  reference  point  along  the 

8 


437 


possibility-probability  continuum"  (p. 1311)  and  claims 
there  is  no  ontological  basis  for  the  dichotomous,  yes- 
no  decision  of  prevailing  practice.    He  calls  for  the 
use  of  "mor«  tolerant"  80%  confidence  levels  in  social 
science  research  (p.  1310/  and  the  reporting  of  one  or 
more  effect  sizes  as  the  primary  product  of  research 
inquiry.    The  choice  of  <*  <  0.10  represents  a 
compromise  between  convention  and  Cohen's  position 
which  allows  for  increased  power  and  yet  provides  a 
reasonably  stringent  test  of  statistical  significance. 

Cohen  (1988,  1990)  also  recommends  power 
analysis.     The  Stone  (1984)  study  was  the  more  recent 
of  two  advance  organizer  meta-analyses  and  more 
consistent  with  advance  organizer  theory.    The  average 
effect  size  of  0.66  for  retention  from  this  study 
represents  an  intermediate  estimate  for  the  effect  of 
instructional  organizers  on  retention  (Kenny,  1990)  and 
was  used  in  this  analysis.     For  E.S.  =  0.66,  n  =  20 
(for  each  treatment  group)  and  <*  <  0.10,  power  (1-jQ)  is 
0.78  and  /3  error  is  0.22.     Cohen  (1988)  recommends  a 
minimum  standard  for  1-0  of  0.80.     Therefore,  based  on 
the  stone  meta-analysis,  this  study  is  slightly 
underpowered. 

Exploratory  Data 

In  addition  to  the  posttests  used  to  measure 
learning,  the  subjects  completed  a  one  page  Program  Use 
Survey  which  provided  a  self  report  of  their  use  of 
such  program  options  as  the  replay  feature,  the  library 
and  the  glossary.    The  time  each  subject  spent  on  the 
program  was  also  measured  and  correlated,  using  Pearson 
r  product  moment  coefficients,  with  posttest  scores. 
Finally,  every  third  subject  from  each  treatment  group 
and  block  (n  =  18)  was  asked  to  participate  in  a  semi- 
structured  interview  with  the  author  after  completing 
the  second  posttest.     These  interviews  were  intended  to 
elicit  a  more  detailed  reflection  .£  the  par^  :ipants' 
use  of  the  CBIV  program  and  of  the  instructional 
organizers. 


Results 

Results  for  descriptive  statistical  data  are 
displayed  in  Table  1  and  for  the  MANOVA  in  Table  2. 
Hypothesis  1  held  that  the  use  of  a  PGO  would  result  in 
a  1-igher  mean  score  on  a  measure  of  immediate  recall 
(Posttest  1)  than  would  the  use  of  either  an  FGO  or  an 
AO.    This  hypothesis  was  not  supported.    The  PGO 
treatmei  c  resulted  in  a  lower  mean  score  on  the  first 
posttest  (18.05)  than  did  the  AO  treatment  (18.857). 
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The  difference  was  not  statistically  significant 
[  J  0.807 1   <  qw  (3,   120)  -  1.77]. 

The  PGO  treatment  also  resulted  in  a 
substantially  lower  mean  score  on  the  first  posttest 
(18.05)  than  did  the  FGO  treatment  (20.9).  Further, 
the  difference  was  statistically  significant  at  the  p  < 
0.01  level.     [ | 2 . 85  I  >  q  w  (3,   60)  -  2.622]. 


Table  1 


Descriptive  Statistics 


Posttest 

1 

Posttest 

2 

N 

M 

SD 

M 

SD 

PGO 

20 

18.05 

3.09 

16.70 

3  .  66 

FGO 

20 

20.90 

2.77 

19.40 

3  .79 

AO 

21 

18.86 

2.99 

18.52 

3  .  30 

Table  2 

Repeated  Measures  Analysis  of  Varianc e 

Tests  of  Hypotheses  for  Between  Subjects  Effe  _ta 


Source 


Treatment 
Order 

Treatment *order 
Block 

Treatment *Block 

Order*Block 

Treatment *Order 

*Block 

Error 


df 

SS 

MS 

F 

p 

2 

94  .08 

47.04 

4.21 

0. 

022*' 

1 

30.33 

30.33 

2.71 

0. 

10? 

2 

19.56 

9.78 

0.87 

0. 

424 

2 

318.63 

159  ,32 

14  .  25 

0. 

0001** 

4 

76.24 

19.06 

1.  70 

0. 

1  67 

2 

2  3..l;4 

11.77 

1.05 

0. 

3  58 

4 

36.00 

9  .00 

0.80 

0. 

529 

3 

480.88 

11.18 

*  p  <  .05 
**  p  <  .01 

Hypothesis  2  stated  that  the  use  of  a  PGO  would 
result  in  a  higher  mean  score  on  a  measure  of  rt -tent: ion 
than  would  the  use  of  either  an  AO  or  an  FGO.  once 
again,  the  hypothesis  was  not  supported  as  the  PGO 
treatment  resulted  in  a  lower  mean  score  on  tho.  second 
posttest  (16.7)  than  did  the  AO  treatment  (18. b2).  The 
difference  was  statistically  significant  at  the  p  < 
0.10  level.      [j  1.84  |  <  q^90  (3  ,   60)  =  1.790],     Tho  IPGO 
treatment  also  resulted  in  a  substantially  iowi<r  mean 
score  on  the  second  posttest  (16.7)  than  did  th«  fc-0 
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treatment.  (19.4).     This  difference  was  statistically 
significant  at  the  p  <  0.01  tevel.     t|2.70|  <  q  w  (3, 
60)   =  2.622]  . 

The  third  hypothesis  asserted  that  the  use  of  an 
FGO  would  result  in  no  greater  effect  on  immediate 
recall  or  retention  than  would  the  use  of  an  AO.  This 
hypothesis  was  not  completely  supported.     The  FGO 
treatment  resulted  in  a  considerably  higher  mean  score 
on  the  first  posttest  (20.9)  than  did  the  AO  "treatment 
(18.857).     This  difference  was  statis' ica  ly 
significant  at  the  p  <  0„0t;  level.     [  J  2  . 043  J   «  q  95  (3, 
86)  =  2.0367],     However,  whilt   the  FGO  treatment" also 
resulted  :.n  a  higher  mean    core  on  the  second  posttert 
(19.4)  than  diu  the-  AO  treatment  (l>i.r24),  the 
difference  was  not  statistically  signif  icant  ['0.877  J  < 
q90  (J,   1/0)   -  1.77]. 

Anal/:.*  is  of  Explc  ratory  Data 

Crosstabulations  of  the  comparison  between 
treatments  by  survey  subques  ion  displayed  no  patterns 
indicating  a  differential  use  of  the  CB:;  /  program  which 
miooi   explain  the  difference  ii   posttest  moans  amor/; 
treatments.     Further,  analysis  o?  variance  of  the  .aenn 
survey  question  scores  showed  no  statistically 
significant  difference  between  the  three  organizers. 

Nor  was  there  an  apparent  difference  among  inn 
three  treatment  groups  't;  the  aiu.Aiiit  of  time  l  iken 
doing  the  program.     Pearler*  r  corr  tlafion  coet:  cient. 
a^-e  reported  in  Table  3.     None  werr  statistically 
significant. 

Ta  le  3 

Pearson  r  Correlation :         -  -   Po^'ttt^jt.  re-j, 

Treatment  V  me 

■c  r- 


AO 

Immediate 

~o . 

"  0.154 

AO 

Retention 

0. 

04  5 

t    R  4  5 

FC.) 

Immediate 

J . 

1  87 

0.1  .1 

FGn 

FC;Centio- 

-0. 

2  (j 

0.10/ 

POO 

i  iirmediate 

-0. 

12  2, 

■  ^oy 

Retention 

)  '53 

Als-: with  ' he*  ^xcrpt.,  ■  -  tlu-  --Li  i*-)  *<  iuf  uc^.*e<Mi  1'^ 
y-'jO  posttcl  i   and  t"me  4  >  u*s  .  ^      i onti))  ij »  wa*- 

always  neya^lv*  .     As  w-u    ,       nv-    t^tai'*  1  ^>  <ly  -  i  *  '  f 


individual  who  did  not  complete  either  case  study. 
Elimination  of  her  scores  changes  the  correlation 
between  time  taken  and  the  FGO  posttest  1  scores  to  - 
0.153  and  while  tne  coefficient  for  posttest  2  drops  to 
-0.27.     It  appears  that  those  who  took  more  time  doing 
the  program  tended  on  average  to  score  lower  on  the 
posttests. 

Finally,  analysis  of  the  interview  data  revealed 
a  number  of  trends.     Perhaps  the  most  striking  trend 
was  that  the  guided  discovery  approach,  which  required 
the  learner  to  make  nu< fling  assessment  and  intervention 
choices  and  rated  them  on  i  l>eir  decisions,  tended  to  be 
distracting  and  disorienting  for  most  subjects. 
Subjects  had  problems  with  the  way  these,  choices  were 
rated,  on  the  amount:  of  information  provided  in 
response  to  these  answers  and  the  manner  in  whieti  it 
was  given. 

The  interview  data  also  revealed  that  the  study 
was  able  to  provide  conditiom   under  which 
instructional  organizers,  particularly  AO's  might  >e 
expected  to  be  effective.     Most  participants  reported 
the  lesson  material  to  be  unfamiliar  and  moderately- 
difficult.     The  effect  of  the  different  organizers  on 
the  subjects'  encoding  process  is  less  clear.  Most 
found  the  AO  treatment  irrelevant.     The  FGO,  on  the 
other  hand,  was  found  to  be  a  useful  orientation  tool 
by  many  study  participants.     It  proved  particularly 
helpful  as  a  guide  to  assessment  ar.d  intervention 
decisions.     That  the  PGO  treatment  tended  to  influence 
encoding  is  less  clear  from  interview  responses.  Somo 
reported  its  usefulness  while  others  found  it  an 
interference.     Several  had  wished  to  alter  its  us<;  to 
suit  their  own  learning  approach. 

Finally,  the  cuestion  of  whether  or  not  the 
restriction  on  note -taking  was  a  confounding  facto  is 
also  unclear.     Sliqhtly  more  than  hnlf  those 
interviewed  indicated  that  they  found  this  an 
interference  while  the  others  stated  it  wat  not  a 
problem.     Those  who  found  tie  restriction  an 
interference  wanted  the  notes  to  alio*  them  to  review, 
either  for  future  testing  or  for  use  of  Uu  information 
cm  the  job.     Those  who  did  not  indicated  that.  It.ey 
would  not  have  taken  notes  regardless  ->r  that  they  were 
poor  note-takers  and  did  not  tend  to  maKt   -jood  use  of 
their  nov.es. 
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Discussion 


The  first  arid  second  hypotheses  predicted  that 
learners  receiving  a  PGO  -  operationalized  as  the  more 
generative  treatment  in  this  study  -  would  outperform, 
on  average,  those  who  received  either  an  AO  or  the 
equivalent  FGO  on  tests  of  both  immediate  recall  and 
retention.     Confirmation  of  these  hypotheses  would 
provide  evidence  that  Wittrock's  (1974)  generative 
learning  hypothesis  could  be  applied  to  the  use  of 
instructional  organizers  with  CBIV.    However,  analysis 
of  the  results  indicated  no  significant  effect  in  favor 
of  the  PGO  treatment  on  the  mean  scores  of  either 
posttest.     In  fact,  the  PGO  treatment  group  had  the 
lowest  mean  score  in  every  comparison.     The  results  of 
this  study,  then,  provide  no  evidence  to  support  the 
application  of  the  generative  learning  hypothesis  as 
operationalized  as  a  PGO.     Unquestionably,  the  filled- 
in  version  of  the  same  organizer  was  the  most  effective 
technique. 

The  third  hypothesis  predicted  no  difference 
between  the  FGO  and  AO  treatments.     While  not  expected, 
evidence  in  favor  of  the  AO  treatment  might  have 
provided  support  for  advance  organizer  theory  and  for 
the  effectiveness  of  this  technique  when  used  with 
CBIV.     The  AO  fell  somewhere  in  between  the  other  two 
techniques.     As  predicted  by  theory  (Mayer,  1979a, 
1979b) ,  the  AO  in  this  study  was  not  effective  in 
aiding  immediate  recall.     However,  neither  was  it  the 
most  effective  in  assisting  retention.  While 
significantly  more  effective  than  the  PGO  treatment,  it 
was  less  effective  (though  not  statistically 
significantly  so)  than  the  FGO  in  aiding  retention. 

These  results  raise  two  issues.     First,  why  would 
the  same  organizer  be  much  less  effective  when  the 
learner  had  to  seek  answers  in  the  CBIV  program  to 
complete  it  than  when  the  information  was  provided? 
Second,  why  was  the  AO  not  more  effective? 

Exploratory  quantitati'  e  analysis  indicated  no 
substantive  differences  between  treatment  groups  which 
might  answer  these  questions..     There  was  no  evidence  of 
the  differential  use  of  the  program  features  nor  of  any 
effect  on  time  on  task  for  any  treatment  group.  As 
well,  examination  of  the  organizers  revealed  that 
extraneous  note-taking,  a  problem  in  the  previous  study 
(Kenny,  Grabowski,  Middlemiss  &  Van-Neste-Kenny ,  1991), 
*?a.5  controlled  in  this  experiment. 
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Was  the  generative  activity  truly  generative? 


The  most  likely  explanation  for  the 
ineffectiveness  of  the  PGO  was  that,  ?s  operational ized 
in  this  study,  it  did  not  constitute  a  generative 
activity.    As  in  the  previous  study  (Kenny,  Grabowski, 
Middlemiss  &  Van-Nest e-Kenny,  1991) ,  completion  of  the 
PGO  remained  a  problem.    Twenty-five  percent  (five  of 
twenty)  of  the  PGO  group  left  five  or  more  blanks 
incomplete  and  ten  percent  left  more  than  ten  spaces 
incomplete.     The  PGO's  were  also  scored  to  determine  if 
they  were  being  completed  correctly.    The  average  score 
was  21.6  of  a  possible  38,  indicating  that  the  subjects 
completing  the  PGO  were  having  difficulty  determining 
what  went  in  the  blanks.     From  these  two  studies,  it 
appears  that  learners  may  need  further  assistance  in 
using  this  technique,  in  the  form  of  training  more 
clear  categories  on  the  organizer  or  some  form  of 
feedback  in  order  to  make  use  of  this  technique. 
Indeed,  Hawk,  Mc?jeod  &  Jonassen  (1985)  do  recommend 
that  the  instructor  review  the  PGO  periodically  with 
the  students  and  verify  its  correct  completion  at  the 
end  of  the  unit.     This  was  not  possible  in  this  study 
and  would  have  added  a  practice  factor  as  an 
alternative  explanation  to  the  expected  generative 
effect.     However,  it  is  feasible  in  CBTV  to  provide  the 
PGO  on  screen  (rather  than  on  paper)  and  to  have  it 
corrected  electronically. 

The  PGO  may  also  have  constituted  an  additional 
activity  in  an  already  generative  situation  and  may 
have  actually  contributed  to,  rather  than 
counteracting,  cognitive  overload.    The  interview  data 
indicate  that  learners  had  considerable  difficulty  with 
the  structured  discovery  approach.     It  is  this  aspect 
of  the  program  design  which  appears  to  provide  the 
strongest  explanation  of  the  superiority  of  the  FGO. 
The  learners  were  repeatedly  faced  with  making  nursing 
assessment  and  intervention  decisions.     They  were 
unable  to  proceed  unless  their  decisions  were 
appropriate.    To  make  the  correct  decisions,  they 
needed  information  and  some  of  this  was  contained  in 
the  blanks  on  the  FGO  (but  left  out  to  be  filled  in  by 
the  learner  in  the  case  of  the  PGO) . 

Interview  comments  about  the  FGO  almost  all 
focused  on  its  usefulness  in  "getting  through"  the 
assessment  and  intervention  segments.    They  saw  it  as  a 
form  of  cheat  sheet.     The  FGO,  then,  appears  to  have 
reduced  learner  confusion  and  disorientation  by  making 
it  easier  to  proceed.     It  reduced  cognitive  overload  by 
directly  indicating  some  assessment  and  intervention 
choices  and  providing  clues  to  others.     The  subjects  in 
this  group  strongly  outperformed  those  in  the  other 
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treatment  groups  on  both  posttents.    Thus,  it  seems 
reasonable  to  suggest  that  the  FGO  may  have  allowed 
those  subjects  receiving  this  treatment  to  concentrate 
more  fully  on  learning  the  material  while  the  subjects 
in  the  other  two  groups  had  to  expend  more  effort 
merely  getting  thrc   jh  the  program. 

The  AO  clearly  did  not  contain  the  information 
needed  to  proceed  through  these  segments.     It  was  not 
meant  to.     Its  purpose  was  to  prepare  the  learner's 
cognitive  structure  to  receive  the  information  in  the 
program  and,  thus,  to  aid  retention.     The  PGO,  would 
only  have  contained  the  needed  information  for  the 
assessment  and  intervention  decisions  if  the  subjects 
in  that  group  had  been  filling  out  the  form  correctly 
as  they  proceeded.     This  assumes  that  they  were  able  to 
determine  what  went  into  the  spaces  provided  before 
reaching  assessment  or  intervention  selection  screens. 
It  is  likely  that  the  participants  were  not  able  to  do 
so.     The  details  were  provided  after  the  assessment  and 
intervention  choices  by  means  of  a  review  of  the 
sequence  and  analysis  by  an  expert  nurse.     Like  the  AO 
treatment  group,  the  only  information  that  they  had 
available  was  what  they  remembered. 

The  members  of  both  the  PGO  and  AO  treatment 
groups  were  faced  with  confusion  and,  occasionally, 
frustration  created  by  this  specific  program  feature 
and  their  particular  organizer  appeared  to  be  no  h^lp. 
To  make  matters  worse,  while  those  in  the  PGO  group 
were  coping  with  this  difficult  process,  their 
organizer  was  asking  them  to  carry  out  an  additional 
task.     It  may  have  produced  cognitive  overload  by 
asking  the  learners  to  expend  more  mental  effort  on 
this  additional  activity  when  they  were  already  engaged 
in  a  difficult  task.    Consequently,  it  is  doubtful  that 
they  were  able  to  sufficiently  focus  on  the  PGO  and  to 
generate  their  own  meaning  from  it.     This  may  explain 
the  relatively  low  scores  (an  average  of  21.6  out  of 
38)  on  the  organizer. 
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Was  the  advance  organizer  used  according  to  theory? 

The  second  issue  pertained  to  the  apparent 
ineffectiveness  of  the  AO.     While  the  AO  in  this  study 
was  more  effective  than  the  PGO,  it  was  consistently 
less  eifective  than  the  FGO.    A  frequent  criticism  of 
advance  organizers  is  that  it  is  not  clear  how  to 
construct  them  (e.g.  Barnes  &  Clawson,  1975,  Hartley  & 
Davies,  1976).     This  is  disputed  by  Ausubel   (1978)  and 
Mayer  (1979a)  who  stressed  that  these  reviews  were  not 
sufficiently  sensitive  to  theory.    Thus,  for  example, 
an  AO  must  be  presented  at  a  higher,  more  abstract 
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level  and  in  advance  of  the  material-to-be-learned.  It 
can  be  expected  to  facilitate  retention  rather  than 
immediate  recall  and  to  be  effective  only  if  the 
material-to-be-learned  is  difficult,  unfamiliar  to  the 
learner  or  disorganized.     Is  it  possible  that  the  AO 
used  in  this  study  was  constructed  poorly  or  used  in  a 
instructional  situation  in  which  it  could  not  be 
expected  to  facilitate  learning? 

The  specifications  of  Ausubel  (1963 ,   1978)  and 
Mayer  (1979a)  were  followed  closely  in  constructing  the 
AO  used  in  this  study.     The  organizer  discussed  two 
higher  level  (more  abstract)  topics,  the  nursing 
process  and  collaborative  problem-solving.  Both 
processes  were  carefully  adhered  to  in  the  nursing  care 
portrayed  throughout  the  program  and  provided  the  basic 
sequence  of  each  case  study.     In  neither  case  study, 
however,  were  the  nursing  process  nor  collaborative 
problem-solving  specifically  discussed,  that  is,  the 
organizer  clearly  is  presented  at  a  higher,  more 
abstract  level  than  the  CBIV  program  material  and 
contains  no  specific  content  (the  actual  cardiac 
material)   from  the  CBIV  program.     It  was  presented 
before  the  subjects  did  the  CBIV  program. 

As  well,  the  subjects  who  participated  in  this 
study  had  neither  been  trained  nor  had  worked  in  the 
cardiac  specialty.    The  interview  data  confirmed  that 
they  were  generally  unfamiliar  with  the  program 
material.     They  also  rated  the  material  as  moderately 
difficult.     Mayer  (1979a)  also  stipulated  that  an  AO, 
to  be  effective,  must  be  capable  of  generating 
relationships  in  the  materiax-tO'-be-learned  and  of 
influencing  the  learner's  encoding  process.     The  first 
point  is  answered  in  the  construction  of  the.  various 
organizers.    All  have  clear  structures  and  were  judged 
capable  of  generating  such  relationships . 

The  second  point,  while  also  a  matter  of  judgment 
by  the  instructor  or  instructional  designer,  can  only 
be  truly  decided  after  the  fact.     The  analysis  of  the 
posttest  mean  scores  appears  to  indicate  that  the  PGO 
had  little  facilitating  effect  (and  may  have  impeded 
encoding)  while  the  FGO  had  the  strongest  influence. 
The  effect  of  the  AO,  which  fell  somewhere  in  the 
middle  of  the  two  -ersions  of  the  graphic  organizer,  is 
difficult  to  determine  from  this  evidence.  The 
interview  responses,  however,-  cast  some  doubt  about 
whether  it  did.     The  AO  taught  (or  reviewed)  the  theory 
of  nursing  process.     Two  graduate  students  in  this 
treatment  group  .noted  that  the  organizer  stimulated 
them  to  think  about  this  process  while  doing  the 
program,    one  junior  failed  to  see  the  use  or  the  AO 
but # remembered  its  content  as  familiar  which  may 
indicate  it  had  a  subconscious  effect.     The  rest  saw  no 
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connection  between  the  program  and  organizer  material* 
For  them,  any  influence  on  encoding,  therefore,  would 
have  had  to  be  quite  subtle. 

In  summary,  perhaps  the  best  explanation  for  the 
effectiveness  of  the  FGO  treatment  stems  from  the 
structured  discovery  approach  of  the  CBIV  program 
design  which  appeared  to  cause  considerable  confusion 
and  disorientation  for  the  study's  participants.  The 
FGO  contained  specific  information  which  may  have 
allowed  the  subjects  in  that  treatment  group  to  more 
successfully  navigate  these  sequences  and  to 
concentrate  more  fully  on  learning  the  content  of  the 
program.     The  AO  and  the  yet-to-be-completed  PGO  did 
not  contain  such  information  and  likely  did  not  provide 
any  assistance.    The  PGO  may  have  even  contributed  to 
cognitive  overload  by  presenting  an  extra  demand  upon 
the  learners1  mental  resources  at  a  time  when  they  were 
already  faced  with  a  difficult  task  and  effectively 
interfered  with  their  learning  rather  than  assisting 
it.     These  results  demonstrate  that  instructional 
design  choices  about  which  such  techniques  to  include 
with  instruction  provided  by  CBIV  depend  on  context. 
Rather  than  depending  on  one  most  effective  technique 
or  instructional  theory  for  all  circumstances, 
instructional  designers  may  have  to  take  into  account 
the  specific  design  attributes  of  the  program  in 
question  and  choose  accordingly. 


Recommendations  for  Further  Research 


It  was  assumed  for  the  purposes  of  this  study 
that  participatory  pictorial  graphic  organizers  (PGO) 
wore  a  form  of  generative  activity  and  would  be  more 
effective  in  assisting  students  to  learn  and  retain 
information  from  a  CBIV  program  than  either  its  FGO 
version  or  an  AO,  forms  which  are  dependent  on  their 
nominal  characteristics  for  any  effect  on  learning. 
This  prediction  was  not  supported  by  the  results.  The 
participatory  form  was  the  least  effective.  One 
explanation  is  that  the  PGO  was  not  a  generative 
activity  as  presumed.     To  test  this  hypothesis,  further 
research  could  be  conducted  comparing  the  PGO  to  other 
instructional  techniques  demonstrated,  or  postulated, 
to  be  generative,  such  as  pattern  notes  (Jonassen, 
1984,  Fields,   1982)  and  concept  maps  (Novak  &  Gowan, 
1984) . 

Second,  the  results  of  this  study  supported  the 
use  of  the  FGO,  at  least  with  instruction  presented  in 
CBIV  of  a  similar  design  to  the  Cardiac  program.  A 
further  test  of  the  effectiveness  of  this  technique 
could  be  carried  by  comparing  its  use  to  that  of 
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hypermaps  (Jonassen,  1989)  or  to  an  epitome  used  in 
instruction  developed  on  the  basis  of  Elaboration 
Theory  (Reigeluth  and  Stern,  1983)  as  suggested  by 
Rezabek  and  Ragan  (1989).    Further,  this  contrast  could 
be  conducted  with  hypermedia  of  varying  structure,  such 
as  a  CBIV  program  developed  using  Elaboration  Theory, 
one  organized  in  modules  like  that  used  in  this  study 
or  even  a  program  featuring  unstructured  hypertext. 

Jonassen  (1989)  also  suggested  that  the  less 
structured  the  hypertext  is,  the  less  likely  are  users 
to  integrate  what  they  have  learned  into  their  own 
knowledge  structures.    He  proposed  that  if  the  material 
can  be  organized  to  replicate  content  or  expert 
knowledge  structures,  then  it  would  better  provide  the 
needed  anchors  for  re-structuring  the  learner's 
knowledge.     In  fact,  in  this  study,  the  AO  and  the  PGO 
were  advanced  as  possible  methods  to  provide  such 
anchors  by  reflecting  the  higher  level  content 
structure  of  the  material-to-be-learned.  The 
assumption  was  that  learners  might  lose  track  of  any 
such  structure  provided  by  the  program  when  using 
various  program  options.    However,  in  this  study,  the 
subjects  followed  the  program  structure  quite 
carefully.     The  comparison  of  the  use  of  the  FGO  to 
other  techniques  such  as  hypermaps  in  both  structured 
and  unstructured  hypertext  versions  of  the  same 
material  might  shed  further  light  on  how  these 
techniques  facilitate  learning. 

Third,  research  on  the  use  of  diagrams  in 
instruc  tion  (Winn  &  Holliday,  1982)  has  indicated  the 
usefulness  of  including  small  pictures  to  teach  concept 
identification.     The  graphic  organizers  in  this  study 
make  use  of  a  number  of  such  pictures.     For  instance,  a 
cartoon  depiction  of  a  heart  with  an  injured  area  is 
used  to  indicate  that  one  case  study  deals  with  a  heart 
attack  while  a  similar  cartoon  represents  congestive 
heart  failure.     Similarly,  a  drawing  of  a  stethoscope 
is  used  to  indicate  sections  of  the  organizer 
pertaining  to  nursing  assessments  and  a  sketch  of  a 
magnifying  glass  to  denote  sections  focusing  on 
observations.     What  is  the  role  of  such  pictures  in  the 
pictorial  graphic  organizer?    Do  they  add  something  or 
does  the  organizer  depend  on  its  diagrammatic  elements 
such  as  the  placement  of  headings  and  boxes  for  its 
effect?    One  relatively  simple  study  would  involve  the 
comparison  of  the  use  of  the  FGO  with  pictures  to  itn 
equivalent  form  with  the  pictures  removed. 

Also,  Wittrock  and  Alesandrini  (1990)  sought  to 
determine  how  learners  use  their  analytic  processing 
ability  (ability  to  abstract  a  common  dimension  across 
stimuli)  and  holistic  processing  ability  (ability  to 
construct  a  whole  from  information  about  its  parts) 
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differently  under  varying  conditions.     In  the  previous 
study  (Kenny,  Grabowski,  Middlemiss  &  Van  Neste-Kenny, 
1991) ,  it  was  hypothesized  that  the  graphic  organizer 
would  stimulate  both  analytic  and  holistic  processing. 
While  no  relationship  was  found,  the  study  also  used  a 
small  sample  size.     The  interaction  of  diagrammatic 
techniques  such  as  the  graphic  organizer  with 
individual  ability  and  preference  needs  further 
investigation.     For  instance,  as  Jonassen  (1989) 
stressed,  techniques  such  as  hypermaps  depend  on 
spatial  skills  that  a  learner  may  lack  and  should  be 
used  with  caution.     Is  this  true?    What  spatial  skills 
are  needed  to  effectively  process  a  hypermap  or  a 
graphic  organizer?    How  would  a  learner's  holistic 
processing  ability  (ability  to  construct  a  whole  from 
information  about  its  parts)  relate  to  these  skills? 

Finally,  the  note-taking  requirement  of  the  PGO 
raises  the  question  of  structured  versus  unstructured 
note-taking.     Do  some  students  take  good  notes  and 
others  not?    Would  students  benefit  from  training  in 
note-taking?    Are  structured  note-taking  techniques  a 
generative  activity  or  ro  they  interfere  with  the 
learner's  generation  of  meaning  by  being  too 
constraining?    There  are  clearly  a  number  of  research 
questions  about  note-taking  suggested  by  this  study 
which  could  be  investigated.    An  obvious  extension  of 
this  study  would  be  the  comparison  of  constraining 
learners  to  the  structure  provided  (as  was  done  with 
the  PGO  treatment)  to  allowing  them  to  use  the 
structure  as  a  guide  which  they  could  modify  as  they 
see  fit.     And  last,   it  would  be  informative  to  compare 
a  structured  note-taking  format  such  as  the  PGO  to 
other  note-taking  techniques  such  as  pattern  noting 
(Jonassen,   1984,  Fields,  1982). 
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The  successful  inclusion  of  school  library  media 
centers  in  fully  articulated  networks  involves 
considerable  planning  and  organization  for  technological 
change.  No  model  to  assist  school  systems  in  the 
decision-adoption  process  previously  existed  in  the 
literature.  In  this  study  a  preliminary  model  of  the 
stages  of  school  system  participation  in  library  networks 
was  developed  with  the  major  activities  for  uach  stage 
identified.  The  model  follows  the  stages  in  the 
innovation  adoption  and  diffusion  literature  and  is 
constructed  from  the  study  of  the  literature , 
observation ,  and  informal  interviews .  The  model  is 
composed  of  four  primary  aspects:  technological  support, 
financial  support ,  human  support,  and  activities  and 
applications.  Within  each  aspect,  anticipated  events  in 
each  of  the  three  stages  of  participation  in  a  network 
are  identified. 

The  population  for  the  study  included  school  systems 
in  one  or  more  of  three  types  of  networks:  OCLC  vendors, 
state  regional  multitype  networks,  and  state-wide  school 
networks.  A  random  sample  of  674  school  systems  in  17 
states  with  3,613  school  systems  was  drawn.  A 
questionnaire  based  on  the  preliminary  stage  model  was 
mailed  to  district  level  persons  responsible  for  school 
media  programs,  school  contact  persons  for  the  library 
network,  superintendents ,  or  building  school  library 
media  specialists .  Usable  responses  were  received  from 
362  school  systems  in  13  states. 

Descriptive  statistics  were  used  to  provide  a 
prof i le  of  the  respondents  by  state ,  type  of  network , 
position  of  respondent,  size  of  school  district,  school 
type,  and  level  of  involvement.  Profiles  by  the  reported 
level  of  involvement  in  a  network  were  done  by  state  and 
size  of  the  school  district.  Chi-square  and  gamma 
coefficient  tests  were  performed  on  each  variable  in  the 
questionnaire  in  relation  to  the  level  of  involvement 
reported  by  the  respondent.  Sixteen  of  the  nineteen 
variables  in  the  model  tested  significant.  When  analyzed 
by  type  of  network  (school  vs  OCLC)  11  events  tested  as 
significant;  however,  when  analyzed  by  position  of  the 
respondent ,  size  of  the  school  system,  and  years  in  a 
network  few  events  tested  significantly  different  under 
the  additional  parameters.  As  do  other  models,  this  one 
provides  a  guide  for  users  to  assess  their  own  progress 
in  adoption  of  an  innovation  and  to  plan  for  positive 
action. 
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SCHOOL  NETWORKING  INNOVATION  MODEL 

Phase  I  Phase  II         Phase  III 


TECHNOLOGICAL  SUPPORT 


*  Micro- 
computers 


*  Tele- 
communi- 
cations 

*  Tele- 
phone 


Some 

evidence  of 
use  of 
computers 

No  modem  in 
school 


No  telephone 
in  media 
center 


Periodic  use 
of  computers 


No  modem  in 
media  center 

Efforts  to 

obtain 

telephone 


Regular  use 
of  computers 


Modem  in 
media  center 

Telephone  in 
media  center 


FINANCIAL  SUPPORT 


Inter- 
library 
Loan 

Network 
dues 


None 

budgeted  for 
ILL 

None 

budgeted  for 
dues 


Requests  for  ILL  costs  in 
ILL  budget  budget 


Special 
funds  used 
for  dues 


Network  fees 
and  dues  in 
budget 


*  Network 
activities 


None  for 

networking 

activities 


Special 
funds  for 
network 
activities 


Communi- 
cation 
expenses 
budget 


in 
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HUMAN  SUPPORT 


Meetings 

int ormai 

r  ormai 

urgani Zdiion 

meetings 

meetings 

rormeu 

Inter- 

Casual 

Centralized 

Individual 

library 

requests  for 

requests  for 

requests  for 

loan 

TT  T 

ILL 

TT  T 

ILL 

TT  T 
ILL 

* 

Policies 

Informal 

Preliminary 

Written 

agreements 

policies 

policies 

oeing  rormeci 

LaUci  rial 

resources 

external 

tion  of 

external 

resources 

external 

resources 

resources 

* 

Staff 

No  additions 

Requests  for 

Staff  added 

staff  made 

Media 

* 

System 

Little 

services 

Consulting 

level 

support 

position  at 

services  by 

support 

Formal 

system  level 

network 

meetings 


ACTIVITIES  AND  APPLICATIONS 


*  Serials 


*  Special 
collections 


*  Collection 
develop- 
ment 


Exchange 
lists  of 
serials 

Exchange 
lists  of 
special 
collections 

Awareness  of 
cooperative 
collection 
d  evelopment 


Union  list 
of  serials 


Union  list 
of  special 
collections 


Exploration 
of 

cooperative 

collection 

development 


Union  catalog 
of  serials 

Union  catalog 
of  special 
collections 

Cooperative 

collection 

development 
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coopera- 

Awareness 

of 

Jixpiora  uion 

i  ecnmcai 

tive 

cooperative 

of 

services 

technical 

technical 

cooperative 

cooperation 

services 

services 

technical 

services 

Delivery 

Awareness 

of 

Investiga- 

Use  or 

methods 

material 

tion  or 

material 

delivery 

material 

delivery 

methods 

delivery 

system 

methods 

Teach  use 

Awareness 

of 

Use  of  union 

Regular  use 

of 

external 

lists  taught 

of  external 

external 

resources 

in 

in  reference 

resources  by 

resources 

reference 

skills 

students 

skills 

instruction 

instruction 

*  significant  for  p<.05 

(Kester,  1990,  72-74,  124) 

The  model  serves  as  a  guide  to  understanding  the 
process  and  also  provides  a  pattern  for  other  school 
systems  to  follow  as  they  explore  resource  sharing* 
EventL  in  each  phase  will  differ  depending  upon  local 
support,  but  the  research  supports  the  general 
activities.  Movement  will  not  be  in  distinct  steps  but 
will  be  a  gradual  transition  as  schools  move  into  a 
shared  resources  environment. 

PHASE  ONE 

TECHNOLOGICAL  SUPPORT.  In  the  area  of  technology, 
the  media  coordinator  is  exploring  the  use  of  a 
microcomputer.  He  or  she  may  need  to  go  to  the  computer 
lab  to  use  a  machine ,  but  at  least  one  application 
package  is  being  used  for  library  management.  Examples 
might  be  a  graphics  production  program  such  as  Print  Shop 
or  a  word  processing  program  for  letter  writing.  There  is 
no  telephone  in  the  media  center.  A  call  other  libraries 
or  vendors ,  necessitates  leaving  the  media  center  and 
walking  to  the  office  or  teacher's  phone. 

FINANCIAL  SUPPORT.  Funding  support  for  resource 
sharing  is  non-existent.  There  is  no  budget  appropriated 
for  sending  materials  to  another  library  -  or  even  for 
returning  borrowed  16  mm  films.  Money  from  school  fund 
raisers  (book  fairs,  school  pictures,  sales)  is  used  for 
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mailing  and  photocopying  is  hopefully  absorbed  in  the 
consumable  materials  budget. 

HUMAN  SUPPORT.  Every  community  has  more  than  one 
type  of  library.  In  Phase  One  the  school  media 
professionals  meet  with  each  other  but  rarely  meet  with 
librarians  from  other  types  of  libraries  in  the 
community.  Requests  for  borrowing  from  other  libraries 
are  casual  and  infrequent.  There  is  no  formal 
interlibrary  loan  agreement. 

School  media  specialists  are  aware  of  external 
resources .  They  attend  meetings ,  read  professional 
journals,  and  have  all  read  Information  Power  with  the 
challenge  "To  participate  in  networks  that  enhance  access 
to  resources  located  outside  the  school"  (AASL,  1988 , 
12).  However,  during  Phase  One  of  the  School  Networking 
Innovation  Model,  there  is  no  concerted  effort  at  the 
system  level  to  support  cooperative  activities. 

ACTIVITIES  AND  APPLICATIONS.  In  the  fourth  area, 
evidence  of  sharing  is  just  beginning.  Lists  of  serial 
holdings  may  be  exchanged  between  librarians.  Awareness 
of  holdings  of  other  libraries  is  necessary  before 
regular  requests  can  be  made.  At  this  stage ,  the  patron 
is  either  sent  to  the  other  library  to  read  the  article 
or  possibly  a  phone  call  will  result  in  a  photo  copy  of 
the  article  being  mailed  to  the  school.  Lists  of  other 
special  collections  may  begin  at  this  phase.  Some  schools 
develop  strengths  in  certain  subjects  to  support  unique 
curricular  offerings.  During  this  phase  media  specialists 
are  becoming  aware  of  different  material  delivery 
methods .  Most  school  systems  have  a  couriers .  Other 
methods  of  delivery  include  dropping  materials  off  in  a 
central  location  after  school  or  commercial  carrier 
services. 

The  transition  into  the  next  phase  will  be  uneven, 
depending  upon  circumstances.  School  district  funding  for 
technology  may  show  movement  into  phase  two  while  the 
human  factor  lags  behind.  In  some  locations  human 
networking  may  take  place  since  funding  is  not  a 
prerequisite  to  librarians  meeting  with  librarians  for 
mutual  decisions  to  cooperate  in  certain  activities. 

PHASE  TWO 

TECHNOLOGICAL  SUPPORT.  During  Phase  Two,  the  media 
specialist  is  beginning  to  use  the  microcomputer 
periodically ,  probably  in  the  media  center .  There  is 
still  no  modem  to  use  to  connect  to  external  resources; 
however ,  efforts  to  obtain  a  telephone  line  in  the  media 
center  have  increased.  The  awareness  of  emerging 
technologies    has    intensified    into    investigation  of 
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services  and  prices.  Terms  such  as  retrospective 
conversion,  MARC  record,  computer  interface,  DOS,  print 
drivers,  hard  disk  drives,  laser  printer,  video  overlay 
card,  telefax,  communications  package,  modem,  DIALOG, 
Dow-Jones,  FrEd  Mail,  WeatherLink,  OCLC,  CD-ROM  drive, 
and  video  disc  player  creep  into  media  specialists'  talks 
with  teachers  and  administrators. 

FINANCIAL  SUPPORT.  Requests  for  budgeted  support  of 
interlibrary  loan  and  resource  sharing  are  beginning  to 
appear  in  proposed  budgets.  Documentation  to  support  the 
requests  are  prepared. 

HUMAN  SUPPORT.  As  schools  move  from  Phase  One  into 
Phase  Two,  the  informal  meetings  of  local  librarians  are 
becoming  regular  with  discussions  on  cooperative 
projects.  Requests  for  interlibrary  loan  may  be 
centralized  and  preliminary  policies  for  resource  sharing 
are  being  formed.  A  school  system  may  request  a  system- 
level  supervisor  for  media  services,  if  such  a  position 
is  not  already  established. 

ACTIVITIES  AND  APPLICATIONS.  The  activities  begun 
during  Phase  One,  are  expanded  during  the  next  phase.  The 
lists  of  serials  and/or  special  collections  are  combined 
into  union  lists.  The  concept  of  cooperative  collection 
development  expands  as  professionals  from  multi-type 
libraries  discuss  and  plan  purchases  of  periodicals  and 
materials.  The  group  may  explore  possibilities  for 
centralized  processing  of  materials,  a  savings  in  time 
and  dollars. 

At  one  time,  networks  considered  only  physical 
delivery  of  materials.  The  items  had  to  be  mailed,  sent 
by  commercial  carrier,  state  courier,  or  sent  through  a 
local  shuttle  or  courier  service.  Some  libraries  have  set 
up  a  central  location  -  the  public  library,  for  example. 
Someone  assumes  responsibility  for  picking  up  the  items, 
often  on  the  way  to  or  from  work.  Methods  of  delivering 
interlibrary  loan  materials  are  explored  during  Phase 
Two.  Students  are  taught  to  use  the  serials  lists  from 
other  libraries. 

Tha  advantages  of  resource  sharing  and  cooperative 
collection  endeavors  lead  libraries  into  phase  three. 
Again,  movement  is  usually  uneven.  A  system  may  be  moving 
into  phase  three  with  technological  support  yet  lag 
behind  in  cooperative  activities  with  other  types  of 
libraries.  Movement  is  influenced  by  opportunities  that 
differ  from  community  to  community. 
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PHASE  THREE 

TECHNOLOGICAL  SUPPORT.  As  schools  move  :.i.to  Phase 
Three  of  the  model,  computers  are  used  regularly  for 
correspondence,  sign  creation,  reports,  and  publications. 
The  publications  include  periodic  newsletters, 
bibliographies  for  instruction  or  recreation,  and 
teaching  materials.  In  addition,  the  computer  has  become 
a  standard  tool  for  library  management:  overdue 
notification,  circulation  records  and  reports,  inventory 
of  equipment,  books,  and  audio-visual  materials.  The 
microcomputer  has  been  often  referred  to  as  "another 
member  of  the  media  center  staff." 

The  telephone  and  modem  are  used  for  online  database 
searching,  electronic  mail  services,  and  interlibrary 
loan  requests.  A  telefax  machine  is  used  to  send  and 
receive  interlibrary  loan  requests  for  periodical 
articles.  Reference  services  are  enhanced  with  the  use  of 
newer  technologies  such  as  CD-ROM  encyclopedias  and 
indexes.  In  addition  to  the  word 

FINANCIAL  SUPPORT.  A  school  system  has  moved  into 
Phase  Three  when  the  costs  of  interlibrary  loan 
activities  are  included  in  the  budget  for  media  services. 
The  communication  expenses  and  network  fees  are  supported 
by  the  school  system  budget. 

HUMAN  SOPPORT.  Local  librarians  are  formally 
organized  with  written  policies  concerning  interlibrary 
loan,  cooperative  endeavors,  shared  staff  development 
programs,  and  the  staff  to  accommodate  a  strong  unified 
media  services  program.  Casual  requests  for  interlibrary 
loans  have  resulted  in  written  policies  for  borrowing  and 
loaning  materials  vdthin  the  area.  Concerns  of  teachers 
over  "materials  they  ordered  for  their  school"  and  of 
school  administrators  about  "materials  purchased  with 
school  money"  (taxpayers'  money)  must  be  addressed  in 
these  policies.  For  school  systems  without  media 
supervisors,  the  members  of  community  librarian 
organizations  become  consultants,  advisors,  and  moral 
supporters  with  the  unified  goal  of  providing  information 
to  the  citizens  of  the  community. 

ACTIVITIES  AND  APPLICATIONS.  The  list  of  serials  and 
collections  has  grown  into  a  union  catalog.  It  may  take 
the  form  of  a  CD-ROM  union  catalog  or  an  online  catalog. 

As  the  librarians  in  a  community  become  more  aware 
of  the  holdings  of  other  libraries,  purchasing  decisions 
are  for  cooperative  collection  development.  Library  A 
will  subscribe  to  Journal  Z  and  Library  B  will  subscribe 
to  Journal  Y.  Neither  is  used  frequently  for  pleasure 
reading  but  both  are  indexed  in  the  standard  periodical 
indexes.    Library   C  will   purchase  Reference  Set  W  and 
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Library  D  will  purchase  Ret&rence  Set  X.  In  effect,  these 
four  libraries  have  quadrupled  their  buying  power. 

The  expense  of  technical  services,  primarily 
processing  new  materials,  may  be  shared.  Whether  a  card 
catalog  or  an  online  catalog  is  used,  if  one  site  serves 
several  libraries,  overall  costs  decline.  One  suggestion 
is  for  a  school  system  to  hire  a  paraprofessional  to  work 
in  the  technical  services  department  at  the  public 
library.  Receiving  orders,  verifying  invoices, 
cataloging,  and  preparing  materials  for  circulation  is 
done  at  the  site.  For  media  specialists  this  shifts  time 
spent  on  things  to  time  spent  with  people,  service. 

To  move  materials  from  library  to  patron  requires 
one  or  more  delivery  systems.  As  the  availability  of 
telefax  machines  increases,  copies  of  articles  in 
journals  and/or  reference  books  are  transmitted  quickly. 
Within  a  local  community,  the  only  cost  is  the  phone  line 
and  the  paper  -  and  the  time  for  workers  to  handle  the 
request. 

Students  are  taught  to  include  external  resources 

while  working  on  term  papers. 

Four  areas  that  influence  resource  sharing  have  been 
discussed:  technology,  funding,  humans,  and  activities. 
These  are  not  meant  to  exclude  other  areas,  but  they  have 
been  identified  as  essential.  Research  supports  that 
these  areas  are  indeed  important  to  a  school  becoming  a 
participating  member  of  library  networking. 
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Educational  Computing:  Social  Considerations 


This  symposium  presented  a  collection  of  four 
related  papers  and  viewpoints  about  the  social,  political, 
and  economic  issues  surrounding  the  use  of  computers  in 
schools.  Participants  in  this  session  hoped  to  make  the 
audience  aware  of  some  of  the  major  social  issues 
surrounding  the  use  of  computers  in  education,  cause  them 
to  reflect  upon  the  effects  of  particular  research  findings 
and  applications  currently  ueing  implemented  in  education, 
and  provide  direction  for  those  in  positions  to  implement 
educational  innovations. 

Each  of  five  presenters  provided  an  important 
perspective  on  educational  issues  related  to  the  use  of 
computer  technology  in  schools,  its  conceptualization, 
practices,  effects,  or  implications.  We  hoped  to  provide  a 
collection  of  ideas  that  have  been  traditionally  ignored  in 
teacher  education  and  graduate  studies,  but  beg  to  be 
addressed.  It  is  from  this  perspective  that  we  formed  the 
contents  and  context  of  this  symposium. 

Each  presenter  addressed  a  particular  aspect  of  a 
critical  social  issue  related  to  the  presence  of  computers  in 
the  schools.  These  issues  include  the  teacher's  changing 
role  and  the  effects  of  computers  on  teachers,  students, 
curriculum  and  knowledge;  the  question  of  education  as  a 
marketplace;  copyright  concerns  and  implications  for 
educational  practice;  and  finally,  instructional  design  and 
human  praxis. 
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EDUCATIONAL  COMPUTING  AND  SCHOOL  CHANGE: 
INFLUENCES  ON  TEACHERS'  ROLES  AND  PEDAGOGY 

Dr.  Nancy  Nelson  Knupfer 
Kansas  State  University 


Technological  advancements  have  stimulated  the  widespread  diffusion  of 
microcomputers  across  all  levels  of  our  economy  and  society,  thus  creating  a  demand  for 
substantive  computer  education.  This  computer  education  includes  learning  about 
computers  as  well  as  learning  with  computers. 

Learning  about  computers  changes  the  curriculum  in  an  obvious  way  while 
learning  with  computers  changes  the  curriculum  and  pedagogy  in  ways  that  go  beyond 
the  obvious.  Furthermore,  those  changes  reach  beyond  the  school  curriculum  as  we 
know  it  and  into  the  social,  economic,  and  political  structure  of  our  lives. 

Although  parents,  government,  and  school  officials  recognize  that  students  must 
prepare  for  citizenship  in  a  computer-based  society,  the  mere  acquisition  of  computers  for 
the  schools  is  but  one  small  facet  of  the  much  larger  and  more  complex  task:  the 
implementation  of  this  potentially  significant  innovation  in  education  and  the  consequent 
formation  of  durable  patterns  of  computer  use  and  application  by  students  and  teachers 
long  into  the  future. 

The  careful  planning  and  proper  implementation  of  informed  computer  education 
agendas,  free  from  unreasonable  expectations,  will  determine  the  value  of  this  innovation 
(Goodlad,  1975;  Romberg  &  Price,  1981).  Further,  the  success  of  any  such 
implementation  effort  will  depend  largely  upon  the  teachers  who  determine  the  daily 
school  activities  (Cuban,  1986).  Indeed,  the  personal  and  professional  relationships 
among  teachers,  students,  and  administrator.-  can  have  a  greater  influence  on  the  future 
of  any  proposed  innovation  than  the  most  generous  outlays  of  money  and  machines. 

The  computer's  conquest  of  American  schools  is  more  sweeping  and  complete  than  that 
of  any  technological  innovation  in  memory.  Around  the  country,  principals,  teachers,  and  parents 
point  with  pride  to  the  serried  ranks  of  monitor  screens  in  their  school  computing  labs  while  they 
battle  with  legislators  over  funding  for  more  machines  and  software.  Indeed,  the  question  of 
implementing  computer  technology  has  already  shifted  from  the  problem  of  acquiring  and  findinq 
uses  for  computers  to  the  problem  of  handling  existing  sets  of  systems,  software,  and  computer 
skills  already  in  place. 

The  ironies  of  the  situation  are  remarkable:  a  machine  designed  simply  to  process  and 
transmit  information  has  become  the  tool  of  choice  in  the  school  system's  campaign  to  foster 
literacy,  transmit  culture,  inculcate  basic  civic  values,  and  train  students  for  a  competitive 
economy.  It  is  perhaps  understandable  that  a  machine  touted  as  the  vanguard  of  the  great  new 
age  of  information  processing  and  control  would  be  so  critical  to  yet  another  of  American 
education's  countless  crusades  to  improve  society  (Ravitch,  1988).  And  like  other  educational 
innovations,  the  ongoing  integration  of  computers  into  education  occurs  in  an  atmosphere 
clouded  by  bewilderment,  optimism,  confusion,  skepticism,  prejudice,  and  hope. 

An  accurate  view  of  educational  change  and  implementation  of  innovations 
considers  the  potential  alterations  that  innovations  cause  in  the  school's  learning 
environment  (Berman,  1981 ;  Berman  &  McLaughlin,  1978;  Sarason,  1982).  For 
example,  the  introduction  of  computers  might  change  the  grouping  of  students  or  the 
amount  and  quality  of  interactions  between  students  and  teachers,  or  the  very  pedagogy 
of  lesson  structure.  Further,  recent  research  indicates  that  changes  in  teachers'  roles  are 
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emerging,  which  in  turn  change  their  relationships  with  other  teachers  and  wi*h 
administrators. 

Implementing  computer  technology  in  schools  has  brought  about  some  obvious 
changes  along  with  changes  which  are  more  subtle,  yet  carry  a  full  impact.  These 
changes  affect  individual  lesson  pedagogy  as  well  as  the  overall  curriculum  strategy,  the 
relationship  between  teachers,  the  emphasis  of  inservice  training,  specific  teaching 
duties,  and  changes  in  institutional  structure.  The  subtle  changes  can  be  just  as  powerful 
as  the  more  obvious  changes  concerning  their  social  effects  on  schools. 

It  is  a  commonplace  to  note  that  the  student  lies  at  the  center  of  this  drama,  while  the 
teacher's  role  in  the  acceptance,  implementation,  and  outcome  of  educational  computing  has 
received  less  attention,  even  though  an  overwhelming  body  of  evidence  points  to  the  centrality 
of  teachers  in  the  process  of  educational  innovation.  It  is  most  important  to  realize  that  the 
teachers'  existing  attitudes,  skills,  and  working  habits  will  have  great  influence  on  their 
acceptance,  style  of  implementation,  and  outcome  regarding  educational  computing  (see  Figure 
1).  The  interaction  between  the  teacher  and  the  computer  has  multiple  dimensions  way  beyond 
the  obvious  problems  of  the  machine's  efficacy  with  students.  We  must  ask,  for  example,  the 
following  questions. 

How  does  the  broad  spectrum  of  teachers'  attitudes  about  computers  and  their 
desirability  limit  or  enhance  the  coordination  of  computers  with  the  curriculum?  Do  teachers' 
prejudices  about  computers,  independent  of  their  biases  about  the  social  or  educational 
condition  of  students,  affect  the  way  computers  are  located,  controlled,  and  used  in  schools? 
How  do  teachers'  fears  and  expectations  about  the  impact  of  computers  on  their  workloads,  daily 
routines,  and  control  over  students  and  classtime  affect  their  decisions  about  supervising  and 
grouping  students  for  computer  use?  These  questions,  part  of  what  might  be  termed  the 
sociology  of  educational  computing,  point  directly  to  the  teacher's  role  in  implementing 
computers. 

Social  Interaction  Model  of  Implementing  Computers 

The  current  method  of  implementing  computers  in  the  schools  is  based  on  a 
"social  interaction"  approach  that  stresses  the  careful  integration  of  practical  ideas 
promoted  by  teachers  with  the  directed  support  of  the  administration,  parents  and  local 
community  (Knupfer,  1987).  Together  these  key  people  provide  the  necessary 
practicality  and  the  directed  support  from  outside  forces  upon  which  the  daily  life  of 
classrooms  is  built.  Teachers  who  develop  workable  strategies  for  using  computers  have 
a  stronger  claim  on  the  vital  support  of  administrators  and  parents  who  lack  practical 
experience  with  educational  computing  but  who  want  to  see  it  succeed.  Implementation 
through  social  interaction  hinges,  then,  on  the  ways  that  teachers  transform  the 
opportunities  afforded  by  an  innovation  into  practical,  meaningful  classroom  applications 
of  this  technology  (see  Figure  2). 

The  teachers'  role  in  the  social  interaction  is  crucial,  and  it  depends  largely  on  the 
preconceptions  teachers  bring  to  the  implementation  of  an  innovation.  Their  attitudes 
about  change  in  the  school;  their  social  prejudices  about  race,  class,  and  gender;  their 
assessments  of  the  students'  different  ability  levels  and  how  to  handle  them;  and  their 
sense  of  their  own  professional  status  and  the  uncertainties  surrounding  that  status  all 
affect  they  way  they  treat  educational  computing. 

Further,  as  the  teachers  grope  with  implementing  computers  into  the  schools  as 
we  know  them,  they  realize  that  schooling  itself  is  changing.  This  change  is  brought  about 
not  only  by  demands  of  a  changing  society,  but  also  by  the  very  presence  of  computers  in 
the  schools.  The  computers  enable  the  teachers  to  structure  lessons  quite  differently 
and  in  so  doing,  change  the  entire  process,  focus,  and  outcomes  of  educational 
expectations.  Teachers  are  not  only  directing  their  students  to  realize  different 
educational  outcomes  as  a  result  of  using  computers,  but  they  are  also  engaging  in 
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different  roles  as  students  experience  different  processes  of  learning.  The  teacher's  role 
is  no  longer  focused  on  providing  information  nor  is  it  to  be  a  straight  forward  facilitator  as 
was  believed  in  the  early  stages  of  educational  computing.  Instead  the  teacher's  role  is 
emerging  as  learning  partner  and  collaborator  in  the  educational  process. 
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Education  as  Marketplace 

Howard  Besser 
School  of  Library  &  Information  Sciences 
University  of  Pittsburgh 

The  exponential  rise  in  computer  education  in  the  1980s  is  usually 
attributed  to  the  technological  innovations  which  made  the  microcomputer  an 
affordable  machine.  Yet,  there  were  many  other  forces  which  contributed  to  the 
widespread  adoption  of  computer  literacy  into  the  curriculum.  This  paper 
focuses  in  on  corporate  needs  and  political  aims  which  united  to  convince 
educators  and  much  of  the  general  public  that  we  were  all  suffering  from  a 
previously  unknown  "disease"  (computer  illiteracy)  and  that  curriculum  reform 
was  necessary  before  this  reached  epidemic  proportions. 

This  paper  concentrates  on  the  twin  needs  for  a  trained  workforce  and  to 
keep  our  country  ahead  of  others  in  the  areas  of  economics,  research,  and 
science/technology.  First  it  sets  the  historical  context  by  showing  that  these 
needs  were  driving  forces  behind  the  first  public  education  programs,  through 
early  vocational  training,  to  the  increase  in  science  and  technology  education  in 
the  wake  of  the  launch  of  Sputnik.  It  then  looks  carefully  at  the  moves  towards 
computer  education  in  the  1980s,  looking  both  at  which  groups  of  people  were 
the  leading  advocates  and  what  they  stood  to  gain  by  this,  as  well  as  examining  the 
curriculum  itself  and  whether  in  retrospect  one  could  say  that  it  was  really 
designed  to  meet  such  a  pressing  need  as  that  vocalized  by  the  leading  proponents 
of  computer  literacy  requirements. 

After  a  detailed  historical  examination  of  these  issues,  the  paper  points  to 
the  following  set  of  conclusions: 

Personal  computers  first  appeared  in  the  classroom  during  a  period  of 
major  curricular  reform  and  must  be  viewed  within  that  context.  The  back  to 
basics  movement  of  that  period  was  grounded  in  the  perception  that  educational 
reform  was  necessary  to  prevent  the  United  States^  from  losing  its  competitive 
edge  internationally  (both  economically  and  politically).  Recommendations 
tended  to  stress  math  and  science  instruction,  vocational  training,  and  computer 
literacy. 

Major  advocates  of  computer  literacy  included  corporations,  higher 
education  leaders,  the  military,  the  federal  government,  and  the  scientific 
community.  The  corporate  world  saw  mandatory  computer  literacy  as  a  way  of 
creating  a  more  trained  workforce  through  direct  computing  skills  (such  as  the 
use  of  particular  software  packages),  as  well  as  general  skills  and  habits 
(working  alone  for  many  hours  without  any  human  contact,  responding  to 
linear/multiple  choice  situations,  etc.).  Computer  corporations  saw  this  as  a 
way  to  promote  popular  interest  and  support  for  their  activities,  as  well  as  a  way 
create  a  demand  for  their  own  particular  products.  Higher  education,  the 
military,  the  federal  government,  and  the  scientific  community  all  viewed 
computer  education  as  a  tool  both  for  maintaining  a  competitive  edge 
internationally  and  for  securing  additional  funding  for  their  own  agendas  which 
intersected  with  this. 
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Computers  and  Copyright:  The  Unpopular  Dilemma 


Abstract 

The  increasing  use  of  microcomputers  in  schools  across  the  nation  is  paralleled  by 
a  growing  concern  that  computer  technology  be  used  responsibly.  One  of  the  many 
concerns  associated  with  responsible  use  of  computers  is  the  issue  of  copyright. 
Educators  are,  or  should  be,  particularly  sensitive  to  copyright  issues.  Yet,  the  problem 
of  computer-related  copyright  violations  in  schools  continues  to  be  one  of  national 
concern.  The  purpose  of  this  paper  is  to:  1)  briefly  explore  microcomputer  copyright 
issues  in  the  context  of  educational  use,  2)  outline  considerations  for  educators  interested 
in  taking  steps  to  resolve  some  of  the  problems  associated  with  computers  and  copyright 
issues,  and  3)  raise  additional  social  issues  associated  with  copyright  laws  and  the 
educational  use  of  microcomputers  and  emeiging  computer-based  technologies. 

Computers,  Classrooms,  and  Copyright 

Computers  offer  a  variety  of  instructional  advantages  in  the  classroom,  yet  the 
increasing  number  of  microcomputers  in  schools  across  the  nation  has  generated 
significant  concern  regarding  associated  ethical  and  legal  responsibilities  of  computer- 
using  educators  and  students.  The  implications  which  copyright  legislation  and 
guidelines  have  for  the  judidous  use  of  computer  hardware  and  software  is  an  area  in 
which  concern  is  growing  as  the  number  of  computers  in  schools  increases.  The  misuse  of 
hardware  and  the  illegal  duplication  and  abuse  of  software  in  violation  of  federal 
copyright  legislation  have  risen  to  epidemic  proportions  (Brunner,  1988;  Bullock,  1986; 
Thornburg,  1986),  And  though  the  severity  of  the  copyright  problem  has  been 
documented,  few  solutions  to  the  copyright  quandary  have  been  offered. 

Unfortunately,  computer  technology  has  evolved  faster  than  the  legislation 
which  governs  its  use,  and  current  copyright  laws  and  guidelines  are  somewhat 
ineffective  in  providing  direction  for  a  concerned  public  (Brunner,  1988;  Sacks,  1985; 
Talab,  1986,  White  &  Hubbard,  1988).  It  also  appears  that  the  existing  copyright  laws 
and  guidelines  are  not  widely  acknowledged  by  educators.  Many  concerned  observers 
(ADAPSO,  1984;  Bullock,  1986;  EDUCOM  &  ADAPSO,  1987;  Helm,  1984;  Thornburg, 
1986)  call  for  educators  to  increase  their  awareness  of  copyright  legislation  and 
guidelines,  believing,  perhaps  naively,  that  increased  knowledge  of  the  law  will 
result  in  increased  adherence  to  copyright  legislation.  But  as  Helm  (1985)  asserts, 
when  educators  violate  computer-related  copyright  laws,  "Some  do  so  in  blissful 
ignorance,  others  with  vague  intuitions  of  wrong-doing,  and  regrettably  a  few 
knowingly  and  blatantly  ignore  the  frustrating  restrictions  contained  in  the  copyright 
law''  (p.  2).  Awareness  of  copyright  laws  and  guidelines  may  serve  to  enlighten  the 
ignorant  and  vaguely  uncomfortable  copyright  offenders,  but  few  sources  offer 
suggestions  for  convincing  the  cognizant  infringers  that  their  actions  are  unethical  and 
illegal  according  to  federal  statutes. 

Educators  often  assert  that  their  responsibilities  as  teachers  absolve  them  from 
adherence  to  the  copyright  laws.  Educators  may  claim  that  copyright  violations  are 
justified  by  their  noble  goals  of  providing  the  best  possible  educational  experiences  for 
students.  But  the  use  of  microcomputers  in  educational  settings  does  not  justify  illegal 
actions,  and  teachers  and  students  must  be  held  accountable  for  their  uses  of 
microcomputer  technology. 

Though  copyright  legislation  and  guidelines  pertain  to  the  uses  of  computer 
hardware  and  software  in  businesses  and  homes  as  well  as  in  schools,  the  misuse  of 
computer  technology  in  schools  is  of  particular  concern.  As  Bear  (1986,  p.  114)  notes, 
schools  are  one  of  the  most  powerful  and  pervasive  influences  on  the  ethics  and  morals 


47i  5u2 


of  young  people  and  schools  must  prepare  students  to  "recognize,  comprehend,  and 
evaluate  the  general  impact  of  computers,  as  well  as  specific  issues  involving  the  use  of 
computers/'  Schools,  therefore,  have  an  obligation  to  introduce  students  to  the  legal 
responsibilities  of  computer  use,  including  copyright  laws  and  guidelines*  In  addition  to 
teaching  students  about  the  implications  of  copyright  legislation  for  computer  use, 
teachers  also  are  called  upon  to  serve  as  role  models  who  demonstrate  ethical  and 
appropriate  uses  of  computer  technology  and  who  are  aware  of  and  adhere  to  copyright 
guidelines  in  their  professional  practice  (Bullock,  1986;  Niemeyer*  1996)*  To  further 
complicate  the  situation,  the  dilemma  of  encouraging  educators  and  students  to  abide  by 
copyright  laws  and  guidelines  is  one  which  will  be  exacerbated  by  the  development  of 
new  computer-based  technologies  before  many  of  the  existing  copyright  problems  are 
resolved. 


Though  the  Copyright  Act  of  1976  and  subsequent  1980  amendments  provide  basic 
guidance  for  the  legal  uses  of  computer  hardware  and  software,  advances  in  technology 
already  have  created  a  need  for  additional  and  more  inclusive  copyright  legislation 
and  guidelines.  For  example,  a  variety  of  computer-accessed  informational, 
interactive,  and  multimedia  options  such  as  compact  audio  discs,  video  discs,  video 
tape,  electronic  databases,  and  bulletin  boards  can  be  linked  by  computers  to  provide 
educational  and  informational  opportunities  for  students  and  teachers.  Students  and 
teachers  certainly  can  use  these  types  of  hardware  and  software  legally  and 
advantageously  in  classrooms.  However,  they  also  can  engage  in  electronic  cut-and- 
paste  techniques  to  incorporate  information  directly  from  these  resources  into  their  own 
products.  This  accessibility  of  information  and  resources  not  only  raises  issues  of 
electronic  plagiarism  and  privacy  of  information,  but  also  raises  copyright  concerns. 

Though  some  electronically-accessed  words,  sound,  and  images  are  authorized  for 
use  in  classroom  productions,  access  to  these  sources  of  computer-based  information 
creates  numerous  copyright  problems  that  are  not  as  yet  clarified  in  the  laws  and 
guidelines.  Teachers  and  students  must  be  made  aware  of  the  responsible  uses  of 
computer-accessed  multimedia  and  informational  resources  in  much  the  same  way  that 
they  must  be  made  aware  of  copyright  guidelines  directing  the  use  of  more  traditional 
media  such  as  film,  audio  cassettes,  and  videotape.  While  D'Ignazio  (1990)  stresses 
that  both  teachers  and  students  must  become  aware  of  copyright  issues  arising  from  the 
use  of  multimedia  resources,  he  also  acknowledges  that  current  copyright  laws  provide 
little  clear  guidance  in  dealing  with  problems  created  by  emerging  technologies. 

Computer-related  copyright  issues  relating  to  multimedia  resources  and  emerging 
technologies  may  require  familiarity  with  guidelines  and  legislation  pertaining  to 
existing  laws  addressing  computer,  print,  video,  audio,  and  other  formats  if  student" 
and  educators  are  to    j  the  new  hardware  and  software  in  an  ethical  manner.  But  in 
order  to  use  copyrighted  materials  within  the  law  and  to  teach  students  the  legal 
responsibilities  associated  with  the  use  of  computers  and  multimedia  hardware  and 
software,  educators  will  need  to  constantly  monitor  the  copyright  laws  and  their 
implications  for  the  uses  of  emerging  technologies.  Unfortunately,  the  technologies 
emerge  more  rapidly  and  also  are  more  readily  acknowledged  than  do  the  laws  and 
guidelines  developed  to  oversee  their  use. 


Efforts  can  be  made  to  encourage  teachers  to  abide  by  the  copyright  laws,  but  it 
will  take  the  concerted  efforts  of  the  entire  educational  community  to  resolve  many  of 
the  issues  and  questions  associated  with  the  legal  uses  of  computer  hardware  and 
software.  Not  only  teachers  and  students,  but  administrators,  district  coordinators  and 
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supervisors,  library  media  specialists,  support  staff,  and  parents  must  cooperate  to 
increase  awareness  of  and  adherence  to  existing  copyright  laws  regarding  the  use  of 
computers  in  schools.  But  the  educational  community  has  an  important  responsibility  in 
addition  to  raising  awareness  of  copyright  laws  and  encouraging  teachers  and  students 
to  abide  by  them.  As  Brunner  suggests,  "Society  must  either  uphold  the  current  laws  or 
change  the  present  laws  in  accordance  with  their  beliefs"  (1988,  p.  17).  Those  concerned 
about  the  educational  uses  of  microcomputer  hardware  and  software  may  believe  that 
the  current  copyright  laws  are  inappropriate  in  an  increasingly  technological  society. 
Those  with  this  conviction  should  serve  as  advocates  for  changing  and  clarifying 
copyright  legislation  to  keep  pace  with  technological  innovations.  As  noted  earlier, 
the  copyright  laws  have  traditionally  failed  to  adequately  address  issues  generated 
by  advances  in  technology,  and  this  trend  continues.  Educators  who  are  bewildered  and 
frustrated  by  the  lack  of  guidance  which  current  laws  provide  or  who  believe  that 
educational  uses  of  computer  technology  demand  more  consideration  than  is  currently 
afforded  by  the  law  must  become  actively  involved  in  the  resolution  of  these  and  other 
computer-related  copyright  issues. 

It  is  important  for  teachers  and  students  to  know  and  to  abide  by  existing 
copyright  legislation,  but  it  is  also  important  for  those  who  live  under  the  law  to  voice 
their  needs  and  convictions.  The  generation  of  copyright  law  has  been  an  attempt  to 
balance  the  rights  of  owners  of  copyrighted  works  with  the  needs  of  those  who  use  the 
works,  and  the  act  of  balancing  is  not  a  static  process.  It  is  up  to  the  educational 
community  to  adhere  to  current  laws  or  to  advocate  a  new  legal  balance.  What  must  not 
continue  is  the  blatant  disregard  for  computer-related  copyright  laws  often 
demonstrated  by  educators.  Educators  must  be  responsible  for  their  use  of  existing  and 
emerging  technologies  and  by  their  consistent  words  and  actions  must  encourage  students 
to  do  likewise.  Until  copyright  laws  are  modified,  an  event  for  which  educators  are  at 
least  partially  responsible,  teachers  and  students  must  know  and  abide  by  existing 
copyright  laws  and  their  implications  for  the  responsible  use  of  computer  hardware 
and  software. 

Copyright,  Computers,  and  the  Broader  Social  Context 

The  suggestion  that  educators  should  advocate  changes  in  copyright  laws  raises 
the  legitimate  question  of  for  whose  benefit  copyright  legislation  is  enacted.  As 
indicated  above,  the  original  intent  of  copyright  law  was  to  balance  the  needs  of  those 
who  use  copyrighted  materials  with  the  rights  of  those  who  own  the  copyright  to  the 
creative  endeavors.  The  fact  that  educators  wish  to  use  copyrighted  materials  in  non- 
profit educational  contexts  does  not  grant  them  permission  to  do  so  indiscriminately. 
Though  educators  are  granted  some  latitude  in  the  use  of  copyrighted  materials,  as 
outlined  in  the  Fair  Use  Doctrine  and  other  copyright  legislation  and  guidelines, 
copyright  laws  nevertheless  limit  the  educational  uses  of  materials,  including  the 
educational  uses  of  computer  hardware  and  software. 

Some  of  these  limits  to  the  educational  uses  of  computer  hardware  and  software 
are  quite  apparent  and  are  spelled  out  in  legislation  and  guidelines  concerning  issues 
such  as  the  duplication  of  software,  site  licensing,  and  networking.  However,  the 
limits  to  the  educational  uses  of  computers  that  are  indicated  by  copyright  legislation 
are  often  more  subtle  and  raise  questions  about  the  educational  uses  of  copyrighted 
computer  materials  as  part  of  the  larger  social  context.  For  example,  educational 
organizations  which  adhere  to  copyright  legislation  and  guidelines  would  necessarily 
need  to  allocate  more  funds  for  the  purchase  of  software  than  would  those  who 
illegally  duplicate  software.  Certainly,  software  producers  suffer  financially  when 
educators  pirate  software,  but  what  is  the  cost  to  students,  teachers,  and  society  when 
school  districts  financially  cannot  provide  access  to  copyrighted  computer  software? 
Will  students  in  financially  less  affluent  districts  have  access  to  instruction  and 
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information  provided  through  computer-based  platforms,  and  if  not,  how  will  this 
deficit  impact  society?  Certainly,  inequities  in  funding  have  existed  for  years  in  school 
districts  across  the  nation  and  continue  to  plague  education.  But  the  potential  of 
computer  hardware  and  software  to  provide  both  instruction  and  information  once  more 
underscores  the  issue  of  equitable  access  to  the  tools  of  education  and  the  keys  to 
knowledge,  if  copyright  legislation  prohibits  students  in  less  affluent  districts  from 
access  to  computer-based  instruction  and  information,  arc  the  laws  in  the  best  interest  of 
society?  If  not,  how  should  they  be  modified  to  reflect  the  changing  needs  of  education 
and  society  that  have  been  precipitated  by  advances  in  technology?  And  how  can  the 
needs  of  students  and  educators  be  balanced  with  the  rights  of  those  whose  creative 
efforts  result  in  computer  hardware  and  software  traditionally  protected  by  copyright 
legislation? 

Obviously,  there  are  no  easy  answers  to  these  questions.  Existing  legislation 
suggests  an  ethical  code  of  behavior  for  educators  as  discussed  above,  but  issues 
regarding  the  educational  uses  of  computers  in  the  larger  social  context  generate 
questions  that  are  as  yet  unresolved  and  that  eventually  may  lead  to  a  new  social  ethic 
as  well  as  to  new  legal  guidelines  for  the  educational  uses  of  copyrighted  computer- 
based  materials. 
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Educational  computing  offers  a  wealth  of  opportunities  to  teach  through  the  use  of 
one's  visual  sense.  That  visualization  can  be  used  to  develop  higher-order  thinking  skills  in  a 
way  that  is  quite  different  from  a  text-based  approach  to  teaching  and  learning.  Yet,  the 
contt.it  of  many  computer-based  lessons  and  their  accompanying  images  on  the  screen  often 
neglect  this  potential  mode  of  teaching.  This  paper  reports  a  study  that  investigated  the 
potential  of  the  Logo  programming  language  to  develop  visual  literacy  skill  in  elementary 
school  students. 

PURPOSE 

The  purpose  of  this  research  was  to  investigate  the  potential  of  a  Logo  environment  to 
develop  visual  literacy  and  recognition  of  Escher-type  geometric  constructions  within  second 
grade  and  sixth  grade  students.  Specifically,  the  following  questions  were  asked.  Can 
students  use  higher-order  and  creative  thinking  skills  in  relation  to  using  computers  to  create 
mathematical  tessellations  and  art?  Can  students  create  Escher-like  tessellations  using  Logo? 
Does  this  enhance  their  visual  literacy  and  visual  perception?  Can  students  transfer  that 
knowledge  when  looking  at  the  prints  of  M.C.  Escher  and  describe  the  more  obvious  figures 
and  the  underlying  geometrical  shapes  within  Escher's  prints? 

BACKGROUND 

Through  the  cognitive  and  organizational  processes  of  thinking,  one  develops  literacy 
in  the  visual  and  verbal  domains.  HVisual  literacy  is  the  active  reconstruction  of  past  visual 
experience  with  incoming  visual  messages  to  obtain  meaning"  (Sinatra,  1986,  p.  5).  Visual 
information  is  sent  to  the  brain  where  it  is  interpreted  and  modified  in  light  of  past 
experiences.  This  process  begins  early  in  life  and  precedes  or  develops  concurrently  with  oral 
literacy  (Sinatra,  1986). 

Visual  literacy  is  the  ability  to  not  only  understand  paintings,  photographs,  film,  and 
other  visual  media,  but  also  the  ability  to  communicate  or  express  oneself  with  at  least  one 
visual  medium  (Curtiss,  1987).  It  draws  its  meaning  from  the  way  one  ihinks  visually,  creates 
new  knowledge,  and  uses  thinking  skills  beyond  those  which  are  basic  in  the  approach  to 
learning.  These  skills  often  employ  higher-order  thinking  which  challenges  the  traditional  path 
of  problem  solving  and  applies  different  approaches  to  learning.  These  different  approaches 
call  upon  different  learning  styles  as  each  situation  demands. 

Our  visual  awareness,  which  is  how  we  see,  and  our  visual  perception,  which  is  how 
we  view  objects  of  nature  and  their  relationship  to  the  space  around  them,  help  us  to  seek, 
uncover  and  construct  new  ideas.  For  centuries,  nature  and  its  objects  have  been  viewed  from 
the  constructs  of  Galileo's  Mgeometricizing  physics"  (Jones,  1989,  p.  8).  Within  this  construct 
nature  is  perceived  solely  in  terms  of  its  geometrical  patterns  and  mathematical  physics. 
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Geometry,  for  example,  has  provided  a  systematic  method  of  understanding  shapes  and  their 
transformation  within  space. 

To  have  an  understanding  of  shapes  within  space  is  a  "perceptual  consciousness" 
(Jones,  1989,  p.  124).  An  awareness;  of  spatiality  is  vital  and  important  in  creating  new 
knowledge.  Just  as  nature  has  been  defined  for  centuries  by  scientists  and  mathematicians,  so 
too  have  artists  visualized  objects  in  space  through  their  paintings,  sculptures,  and  other  media. 
They  discover  space  through  artistic  expression  rather  than  mathematical  induction. 

But  artists  and  mathematicians  rarely  mix  their  discoveries.  As  forms  of  expression, 
the  arts  and  mathematics  have  not  been  assimilated  by  artists  or  scientists.  This  is  often  due  to 
the  unwritten  doctrine  that  artists  should  not  practice  intellectuality  in  their  work  (Locher, 
1988). 

Escher,  however,  combined  the  ideas  of  the  empirical  world  as  an  illusion  with 
mathematical  representations  of  that  world.  The  mathematics  represented  the  unifying 
principle  upon  which  a  work  was  constructed  and  arranged  (Thornburg,  1983).  Escher's 
prints  conveyed  his  ability  to  balance  perfect  forms  and  structures  with  the  idea  that  everything 
is  simply  an  illusion  (Locher,  1988).  These  artistic  renderings  of  illusory  geometric  shapes 
rotating  about  a  point  in  space  generate  a  mathematical  beauty  which  has  the  ability  to  please 
the  senses  and  elate  the  mind  of  the  viewer. 

Viewing  Escher's  works,  appreciating  Escher's  vision  and  mathematical  ideas,  and 
being  able  to  describe  what  is  visually  present  requires  conceptual  thinking.  Arnheim  (1974) 
describes  this  form  of  reasoning  as  an  artistic  activity  and  claims  that  the  process  calls  upon 
perception  and  thought  which  are  intertwined  in  an  indivisible  manner.  He  posits  that  our 
senses  employ  impressions  and  experiences  from  which  we  form  perceptions,  which  we  then 
express  through  our  thoughts.  Arnheim  indicates  three  elements  that  are  necessary  for  human 
cognition:  common  characteristics  that  control  perception  in  the  nervous  system  and  its 
reflections  upon  consciousness;  constructing  physical  reality  through  the  senses;  and  the 
properties  inherent  in  media  through  which  cognitive  experiences  take  place  (Arnheim,  1986). 
This  process  of  visual  conception  is  formed  by  one's  intent,  creative  resourcefulness,  and  fresh 
ideas  (Curtiss,  1987).  Through  this  process  of  visual  thinking  we  are  able  to  evaluate  our 
visual  experiences  and  make  visual  statements  regarding  our  perceptions. 

Being  visually  literate  is  tantamount  to  being  educationally  well-rounded  as  a  person. 
This  premise  has  major  implications  for  educators  and  the  curriculum.  According  to  Arnheim 
(1986),  visual  thinking  promotes  a  continuous  link  between  pictures  and  words,  and  between 
the  arts  and  sciences.  These  links  extend  more  broadly  to  encompass  both  language  and 
images,  as  well  as  perceptual  experience  and  theoretical  reasoning.  And  because  the  structural 
properties  of  images,  symmetry  for  example,  drive  the  thinking,  Arnheim  advocates  that 
teachers  use  specific  rules  related  to  pictorial  composition  and  visual  order  when  presenting 
new  geometry  concepts  to  students.  Thus,  students  will  be  encouraged  to  think  in  terms  of 
visual  images. 

Visual  thinking  as  a  primary  medium  of  thought  can  be  defined  as  having  three 
components  which  include  perceptual  images,  mental  images,  and  graphic  images 
(Erikkson,1988).  Perceptual  images  are  the  ways  we  choose  to  see  and  interpret  the  external 
world.  Mental  images  are  the  ways  we  invent,  envision,  create  ideas  to  solve  problem*  and 
integrate  information.  Graphic  images  are  the  visual  codes  and  languages  we  use  to 
communicate  information. 
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According  to  Eriksson  (1988),  the  visual  nature  of  communication  and  the  fact  that 
visual  images  are  the  building  blocks  of  data  require  a  new  language  based  upon  verbal-visual 
cues.  This  requires  educators  and  curriculum  designers  to  emphasize  visual  literacy  skills, 

The  computer  has  the  potential  to  offer  instruction  based  upon  visualization.  For 
example,  images  created  on  the  computer  screen  can  enhance  the  meaning  of  lessons.  In 
addition  particular  software  which  is  based  upon  visualization  may  have  the  potential  of 
helping  students  develop  visual  literacy  skills.  For  example,  Logo  uses  visual  elements  which 
show  students  the  results  of  structured  procedures  that  they  write.  Kmipfer  (1991)  recently 
reported  some  interesting  patterns  of  increasing  cognitive  skill  related  to  spatial  scanning 
among  students  who  used  Logo  during  group  activity.  This  research  investigates  the 
possibility  that  using  Logo  will  help  students  develop  visual  literacy  skills  and  spatial  scanning 
as  it  relates  to  Escher-like  tessellations. 

METHOD 

Subjects 

Four  classes  of  students  participated  in  this  study:  two  classes  of  second  grade 
students  and  two  classes  of  sixth  grade  students,  totalling  110  students.  All  classes  were 
heterogeneously  mixed.  One  teacher  instructed  both  second-grade  classes  and  another  teacher 
instructed  both  sixth-grade  classes.  None  of  the  second  graders  had  experience  with  Logo 
prior  to  this  study,  and  approximately  half  of  the  sixth  graders  had  been  exposed  to  Logo 
during  the  previous  school  year. 

A  control  group  of  126  students  was  used;  again,  there  were  two  classes  of 
second-grade  students  and  two  classes  of  sixth-grade  students.  The  control  group  was  from  a 
different  school  within  the  same  geographical  region  and  consisted  of  a  similar  population  of 
students.  The  control  group  did  not  receive  he  Logo  treatment  and  practice. 

Tables  one,  two  and  three  should  the  breakdown  of  subjects  by  treatment  and  control, 
grade  level,  gender,  and  racial  group. 

Mateiials 

Pretest  and  Posttest.  A  pretest  and  posttest  were  conducted  using  Escher  prints. 
Two  prints  were  used  for  the  pretest  and  three  different  prints,  which  had  similar  qualities  in 
terms  of  this  study,  were  used  for  the  posttest.  All  prints  had  one  or  two  figures  and  one 
geometric  shape  upon  which  the  print  was  constructed.  The  pretest  and  the  posttest  each 
contained  two  colored  prints  along  with  one  black  and  white  print. 

Software.  LogoWriter  (LCSI)  was  used  as  the  software  with  which  students  practiced 
creating  and  tessellating  shapes.  The  students  created  all  geometric  shapes,  patterns,  and 
tessellations  within  this  software  environment  with  the  exception  of  one  paper  and  pencil 
exercise  that  served  as  an  introduction  to  the  concept  of  tessellations.  Use  of  the  Logo 
language  allowed  the  students  to  work  within  a  microworld  as  Papert  envisioned, 

Escher  Artwork.  Samples  of  Escher  artwork  were  displayed  in  the  computer 
laboratory  where  the  students  had  their  classes.  Also,  a  few  books  on  Escher  were  placed 
within  the  school  library  for  the  students  to  check  out. 
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Table  1.  Subjects  by  Grade  and  Treatment 


GRADE 

TREATMENT 

CONTROL 

TOTAL 

2nd 

48 

55 

103 

6th 

64 

71 

135 

TOTAL 

112 

126 

238 

Table  2.  Subjects  by  Grade  and  Gender 


GRADE 

MALE 

FEMALE 

TOTAL 

2nd 

46 

57 

103 

6th 

70 

65 

135 

TOTAL 

116 

122 

238 

Table  3.  Subjects  by  Racial  Group 


CAUCASIAN 

HISPANIC 

BLACK 

COUNT 

143 

85 

10 

PERCENTAGE 

60.1 

35.7 

4.2 

0 
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Paper  and  Felt-tip  Pens.  One  class  period  called  for  the  introduction  of  tessellations  using 
a  paper-based  activity  in  which  the  students  created  tessellations  and  colored  patterns  within 
them. 

Procedure 

This  study  was  conducted  during  an  entire  spring  semester;  students  used  computers  in  a 
laboratory  setting  for  one  40-minute  session  during  each  week.  Most  of  the  students  worked 
in  preassigned  pairs.  Students  were  individually  pretested  using  two  Escher  prints.  They  were 
asked  to  describe  the  recognizable  figures  and  geometric  shapes  that  they  saw  in  the  prints. 

Next,  the  students  received  instruction  in  creating  shapes  using  the  LogoWriter  software. 
They  were  taught  commands  to  first  create  the  shape  and  then  to  write  a  procedure  for 
constructing  the  shape.  Initially,  shapes  were  limited  to  squares,  triangles,  and  hexagons  so 
that  students  could  tessellate  their  work.  Once  students  mastered  writing  procedures,  they 
were  taught  the  MOVE  command  and  practiced  tiling  their  shapes  vertically  or  horizontally  on 
the  screen.  Finally,  students  were  asked  to  tessellate  their  shapes  and  fill  the  screen  with  the 
tessellations. 

In  accordance  with  research  on  transfer  of  knowledge,  once  the  foundations  were  laid,  the 
students  were  given  instruction  on  Escher  prints  and  the  Escher  style  of  creating  tessellations. 
Approximately  twelve  weeks  into  the  study,  one  researcher  taught  a  lesson  to  familiarize  the 
students  with  Escher  and  the  concept  of  tessellations  within  artwork.  During  a  single  class 
period,  the  students  were  shown  examples  of  Escher  artwork  and  were  able  to  look  for  the 
figures  and  geometric  shapes  within  it.  Both  the  treatment  and  the  control  group  were  taught 
the  Escher  lesson,  but  the  control  group  simply  heard  the  lesson  and  received  no  ability  to 
practice  applying  its  concepts.  On  the  other  hand,  after  hearing  the  presentation  and  seeing 
some  examples,  the  students  in  the  treatment  group  were  able  to  try  creating  a  tessellation  by 
first  drawing  a  plan  on  a  piece  of  paper  and  trying  to  create  that  plan  on  the  computer. 

During  the  final  three  weeks  of  the  unit  the  sixth-grade  students  who  had  already  mastered 
tessellating  the  computer  screen  with  standard  geometric  shapes,  were  able  to  try  more  a  more 
complicated  task.  Those  students  wrote  procedures  which  pushed  in  and  pulled  out  the  sides 
of  a  square  as  part  of  the  process  of  creating  Escher-like  shapes  on  the  computer.  Figure  1 
shows  an  example  of  this  type  of  work.  The  shape  stands  alone  as  it  was  created  and  then  it  is 
incorporated  in  a  tessellating  pattern.  Students  who  were  able  to  create  more  complicated 
patterns  went  on  to  design  fancier  tessellations  like  the  one  pictured  in  Figure  2. 

The  students  continued  to  practice  tessellating  their  shapes,  printed  the  full-screen 
tessellations  on  paper,  and  drew  Escher-like  figures  on  their  printouts  with  colorful  felt-tip 
pens.  Finally,  the  students  were  posttested  •'"ing  three  additional  Escher  prints  that  they  had 
not  seen  before.  The  posttest  followed  the  same  method  as  the  pretest  and  the  only  variable 
was  fresh  prints  to  avoid  the  recognition  of  previous  viewings.  The  control  group  received  no 
Logo  instruction  and  were  simply  given  the  pretest  and  posttest. 

Teaching  Style.  The  second-grade  students  were  taught  by  one  second-grade  teacher  who 
had  previously  worked  as  a  computer  lab  teacher.  The  sixth-graders  were  taught  by  the 
current  computer  lab  instructor.  Both  instructors  were  knowledgeable  in  the  use  of  computers 
and  LogoWriter.  Their  approaches  to  teaching  with  computers  were  similar,  but  their  teaching 
styles  were  not.  The  second-grade  teacher's  style  was  gregarious,  animated,  and  experientially 
oriented  while  the  sixth-grade  teacher's  style  was  structured,  linear,  and  oriented  toward  rote 
learning. 
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Figure  1 

Shape  and  Tessellation  of  a  Pushed  In  and  Pulled  Out  Square 
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Figure  2 

Shape  and  Tessellation  of  a  Cube 
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Prior  to  beginning  the  Logo  instruction,  both  teachers  were  given  a  script  to  guide  them 
through  the  weekly  lessons.  The  second-grade  teacher  began  by  following  the  script  but 
changed  the  script  in  consultation  with  the  researchers  when  he  felt  the  students  needed  more 
time  to  work  on  understanding  particular  concepts,  such  as  how  to  make  the  turtle  move  or  the 
meaning  of  tessellation.  The  sixth-grade  teacher  followed  his  own  course  of  study  with  the 
unit  and  ignored  the  script  from  the  beginning. 

Analysts 

The  independent  variables  for  analysis  were  Logo  treatment  and  grade  level.  The 
dependent  variables  were  figure  and  geometric  shape  identification.  ANOVA  determined  that 
there  were  no  significant  differences  in  the  pretest  scores  for  either  figure  or  shape 
identification  among  the  student  groups.  The  posttest  scores  were  analyzed  using  a  2  X  2 
ANOVA  (grade  level  by  treatment  group)  and  t-test  between  treatments  within  each  grade 
level.  The  analysis,  which  follows,  allowed  some  conclusions  to  be  drawn  about  the  feasibility 
of  using  Logo  to  enhance  visual  literacy  and  spatial  scanning  skills  which  might  transfer  to  the 
ability  to  recognize  Escher-type  geometric  patterns  of  figures  and  shapes.  The  alpha  level  for 
significance  was  set  at  .05. 

RESUI TS  AND  CONCLUSIONS 

There  were  two  dependent  variables  in  question  for  the  analysis;  figures  and  geometric 
shapes.  Table  4  lists  the  means  and  standard  deviations  for  The  ANOVA  for  figure 
identification  revealed  significant  main  effects  for  both  treatment  and  grade,  but  no  significant 
interactions  (see  Table  5).  One  could  interpret  this  to  mean  that  the  students  who  received  the 
Logo  practice  did  better  than  those  who  did  not  receive  Logo.  In  addition  students  in  the  sixth 
grade  recognized  more  figures  than  did  the  second  grade  students.  However,  there  is  no 
interaction  between  those  main  effects. 

The  ANOVA  for  geometric  shape  identification  revealed  significant  differences  for  the 
main  effects  as  well  as  an  interaction  between  the  independent  variables  of  treatment  and  grade 
(see  Table  6).  This  inaicates  that  sixth  grade  students  identified  significantly  more  shapes  than 
did  second  grade  students,  and  those  in  the  treatment  group  identified  more  shapes  than  those 
in  the  control  group.  Further,  the  older  students  who  received  the  treatment  identified  the 
most  geometric  shapes. 

Student  Attitudes.  Throughout  the  study  the  students  remained  interested  in  the  daily  tasks. 
The  second-graders  experienced  some  frustration,  but  because  the  teacher  was  alert  to  its 
signs,  adjustments  were  made  to  the  lessons  which  allowed  students  to  proceed  at  a  more 
comfortable  level.  The  sixth  graders  demonstrated  various  levels  of  enthusiasm.  Some 
students  worked  ardently,  while  others  did  not  remain  on  task. 

Once  the  students  began  to  be  successful  with  the  tessellations,  their  interest  level 
increased;  this  was  true  of  both  grade  levels.  After  the  lesson  on  Escher  and  his  artwork,  the 
students  showed  a  remarkable  interest  in  the  activity.  Some  second  graders  were  heard  to 
remark  that  they  were  going  to  Mplay  artist8  after  school  and  some  of  the  sixth  graders  checked 
out  books  on  Escher  from  the  library.  Students  at  both  grade  levels  wanted  to  know  where 
they  could  find  other  things  by  Escher.  For  example,  the  researcher  had  shown  the  students  a 
tote  bag,  some  wrapping  paper,  and  some  posters  contained  Escher  prints,  and  the  students 
wanted  to  buy  some  of  those  things  for  themselves.  They  were  also  interested  in  finding  more 
books  about  Escher  at  the  public  library. 
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Table  4.  Means  and  Standard  Deviations  on  Posttest 


FIGURES  GEOMETRIC  SHAPES 


GROUP 

n 

M 

SD 

M 

SD 

2nd  Grade  Treatment 

48 

2.71 

.62 

1.35 

1.02 

2nd  Grade  Control 

55 

2.45 

.86 

.18 

.51 

6th  Grade  Treatment 

64 

2.92 

.42 

2.12 

.88 

6th  Grade  Control 

71 

2.85 

.32 

0.30 

.52 

Table  5.  ANOVAfor  Figure  Identification  (Treatment  by  Grade) 


Source 

D.F. 

Sum  of  Sq. 

Mean  Sq. 

F  Ratio 

F  Prob. 

Treatment  (A) 

1 

1.30 

1.30 

3.95 

.048* 

Grade  Level  (B) 

1 

5.79 

5.79 

17.66 

.000* 

Treatment  X  Grade 

(AB)  1 

.53 

.53 

1.62 

.204 

Error 

234 

76.75 

.33 

Total 

237 

84.37 

.36 

*  =  Significant  at  the  <  .05  level 


Table  6.  ANOVAfor  Geometric  Shape  Identification  (Treatment  by  Grade) 


Source  D.F.     Sumof'Sq.  Mean  Sq.    F  Ratio  F  Prob. 


Treatment  (A)  1  142.19  142.19  254.10  .000* 

Grade  Level  (B)  1  10.42  10.42  18.62  .000* 

Treatment  X  Grade  (AB)    1  6.28  6.28  11.21  .001* 

Error  234  130.95  .56 

Total  237       289.84  1.22 

*  -  Significant  at  the  <  .05  level 

5 ;  5 
40^ 


Based  upon  the  observed  performance  and  attitude  of  the  students  along  with  the  changes  m 
their  ability  to  see  geometric  constructions  within  a  piece  of  artwork,  it  appears  that  using 
Logo  can  be  helpful  in  promoting  visual-spatial  awareness.  This  is  particularly  true  at  the 
sixth  grade  level.  Even  though  the  second  graders  were  not  able  to  create  tessellations  by 
building  shapes  from  scratch  on  the  computer,  they  did  get  the  experience  of  creating  the 
tessellations  within  Logo  by  using  the  shapes  page  and  the  SHADE  command,  so  the  end 
result  was  similar.  They  also  were  able  to  view  the  computer  screens  of  their  peers  to  discuss 
which  ones  tessellated  and  which  ones  did  not. 

Even  these  young  students  were  able  to  improve  their  performance  on  the  posttest.  T-tests 
on  the  total  group  of  second  grade  students  revealed  that  second  graders  in  the  treatment  group 
scored  significantly  better  in  recognizing  both  figures  and  geometries  shapes  (see  Table  7). 

Table  7.  T-Test  between  Control  and  Treatment  Groups  of  Second  Grade  Students 


Group 


Mean    SD       SE       F  Value   2-Tail  Prob. 


Figures 

Control  55        2.45      0.86  0.12 

Treatment         48        2.71      0.62  0.09 


Geometric  Shapes 

Control  55        0.18      0.51  0.06 

Treatment         48        1.35      1.02  0.15 


1.93  0.02 


3.97  0.00 


Significant  at  the  <.05  level 


Students  in  this  study  seemed  to  have  a  positive  attitude  about  this  unit.  While  some 
students  struggled  with  the  assignments  and  were  not  able  to  complete  the  on-screen 
tessellations,  others  created  Escher-lilu  designs,  thus  accomplishing  the  overall  objectives  of 
the  unit. 

In  this  case  the  choice  of  teachers,  the  amount  of  weekly  time  spent  in  the  computer  lab 
and  duration  of  this  study  were  controlled  by  the  school  schedule.  Although  it  is  practical  to 
conduct  a  study  within  the  confines  of  the  daily  school  routine  this  situation  places  limitations 
on  the  research.  It  would  be  helpful,  for  instance,  to  use  the  same  teacher  for  both  grades  to 
eliminate  teacher  variability.  Even  though  this  study  tried  to  control  for  teaching  technique  by 
scripting  the  lessons,  one  teacher  did  not  adhere  to  the  script  and  used  a  teaching  style  that  told 
answers  to  the  students  rather  than  challenging  the  students  to  discover  the  answers.  This 
makes  it  difficult  to  have  absolute  confidence  in  comparisons  between  the  grade  levels. 

Although  schools  tend  to  focus  on  activities  that  use  left  brain  or  language  processes  more 
than  right  brain  processes,  many  students  do  function  very  well,  or  even  better,  using  the 
artistic  element  promoted  by  the  right  side  of  the  brain.  If  Logo  is  valuable  in  aiding 
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development  of  visual  literacy,  then  it  certainly  can  benefit  students  at  the  lower  and  upper 
elementary  grade  levels. 

Further  research  should  be  conducted  using  this  study  as  a  pilot  test.  Such  research  would 
be  most  helpful  if  it  encompassed  a  longer  period  of  time  for  the  activities  in  the  unit, 
particularly  at  the  second-grade  level.  The  second  graders  probably  need  a  full  school  year  to 
accomplish  the  objective  of  learning  to  tessellate  figures  from  scratch  within  the  Logo 
language.  A  longitudinal  study  might  also  include  the  question  of  how  the  combination  of 
computers  and  art  over  time  affects  students  attitudes  toward  the  visual  arts. 
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Channel  One  is  a  twelve  minute  video  news  magazine  which  is  currently  beamed  by  satellite 
to  over  2000  high  schools  in  the  United  States.  Produced  by  Whittle  Communications,  the 
program  attempts  to  bring  current  events  to  students  while  they  are  in  the  classrooms.  The 
controversy  surrounding  the  use  of  this  program  in  schools  stems  primarily  from  the  fact  that  two 
minutes  of  advertising  are  embedded  in  each  twelve  minute  program.  New  York  and  California 
have  banned  the  use  of  this  program  in  the  classroom  on  the  grounds  that  Chris  Whittle  is  selling 
student  auaences  to  advertisers.  (A  serious  challenge  to  the  California  ban  is  pending.)  Without 
discounting  the  importance  of  "Sanctioned  adsH  in  U.S.  classroom,  we  would  like  to  raise 
additional  questions  about  the  meaning  of  this  innovation  in  school  systems. 

Concerns  and  questions  abound  in  any  discussion  of  the  use  of  Channel  One  in  our  public 
schools.  Many  educators  are  concerned  about  the  impact  that  such  an  innovation  might  have  on 
the  students  and  ceachers,  the  curriculum,  and  the  community  itself.  Parents  are  interested  in  their 
children  having  the  best  educational  opportunities,  but  are  questioning  if  this  innovation  represents 
the  best. 

Tliis  symposium  is  based  on  a  research  project  which  was  completed  during  the  Fust  year  of 
the  program's  broadcast.  Six  researchers  with  different  areas  of  expertise  investigated  the  impact 
of  Channel  One  in  varying  aspects  of  schooling. 

An  expert  in  methods  of  empirical  research  conducted  a  study  to  measure  student's  retention 
and  comprehension  of  current  event  facts;  this  research  involved  over  4000  students  in  various 
schools  districts.  In  addition,  a  survey  was  administered  to  students  and  teacheis  at  the  beginning 
of  the  year  and  again  after  a  year  of  viewing  the  program;  the  survey  measured  attitudes  and 
opinions  about  the  program. 

A  professor  who  conducts  and  teaches  case  study  methodology  directed  a  case  study  of 
Channel  One  at  a  participating  site.  The  site  chosen  was  a  small  district,  with  one  high  school 
and  a  junior  high.  Several  teachers  and  students  participated  in  in-depth  interviews  and  the  data 
was  analyzed  qualitatively.  The  report  of  the  case  study  focuses  on  several  aspects  of  the  project, 
such  as  the  diffusion  of  innovation  in  schools,  the  involvement  of  teachers  in  such  innovation,  and 
the  effect  of  an  innovation  on  curriculum  and  teaching  practices.  The  implications  of  the  project 
are  many,  ranging  from  policy  about  involvement  of  private  sector  in  public  education  to  reactions 
and  observations  of  students  and  teachers. 

Another  researcher  who  has  experience  in  policy  issues  examined  the  impact  of  Channel  One 
on  curricular  issues.  When  any  innovation  causes  so  much  debate,  policy-makers  and  decision- 
makers need  to  reflect  on  possible  implications  of  the  innovation  within  schools. 
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An  advertising  professor  and  researcher  will  address  the  impact  of  Channel  One  advertising 
on  the  teen  viewer.  Much  of  the  public  news  and  debate  about  Channel  One  focuses  on  the 
advertising. 

Another  team  member  conducted  a  structural  analysis  of  the  program  itself  to  ascertain  what 
messages  are  being  encoded  in  the  production  and  how  students  are  interpieting  them.  And,  the 
final  member  of  the  research  team  coordinated  the  team's  efforts. 

Specific  focus  questions  directed  the  research  team  as  follows: 

•  Do  students  remember  and  comprehend  current  events  after  watching  Channel  One! 

•  Do  students  remember  and  comprehend  commercial  advertisements  in  Channel  One! 

•  Does  Channel  One  heighten  student  interest  in  current  events? 

•  Does  Channel  One  heighten  student  interest  in  products?  advertised? 

©  In  addition  to  current  event  information,  what  messages  are  embedded  in  the  program? 

•  How  are  students  interpreting  embedded  messages? 

•  What  is  the  impact  of  Ciannel  One  on  lesson  planning? 

•  What  is  the  impact  of  Channel  One  on  curriculum  development? 

•  For  school  boards,  what  are  the  implications  of  selling  target  school  populations  to  advertisers 
in  the  private  sector? 

•  What  are  the  implications  of  private  sector  curriculum  development  for  professional 
educators? 

•  What  influenced  your  decision  to  use  Channel  One! 

•  How  was  the  decision  reached?  Who  helped  make  the  decision? 

•  What  was  the  initial  teachers,  student,  and  parent  reaction? 

•  Who  coordinates  the  use  of  the  program? 

•  Have  the  teachers  been  helped  in  their  use  of  the  materials? 
«»  How  have  students  used  the  information? 

•  What  has  been  the  reaction  of  the  participants? 

•  What  will  influence  the  success  of  this  project? 

•  What  advantages  and  disadvantages  are  seen? 

•  Will  the  project  be  continued? 

•  What  other  use  of  the  equipment  is  made  or  planned? 
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When  is  the  program  shown? 

How  do  students  feel  about  the  program? 

How  do  students  and  teachers  feel  about  the  commercials? 

What  is  the  place  of  so  called  free  and  inexpensive  materials  in  the  classroom? 

What  is  the  function  of  journalism  in  a  free  society? 

What  is  the  responsibility  of  citizens  in  a  democracy  to  fully  inform  themselves? 

Can  the  privatization  of  information  and  discourse  ever  be  compatible  with  public  discourse? 

Should  impoverished  school  budgets  limit  teachers  and  learners  to  so  called  free  sources  of 
information? 
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Introduction 

Life  in  the  information  age  is  predicted  to  be  easier  and  more  productive.  The 
common  use  of  modem  technology,  such  as  the  use  of  robotics,  electronic  information 
retrieval  and  communication,  as  found  in  teleconferencing  and  electronic  mail,  and 
in  voice  and  computer  controlled  simulated  environments,  will  enable  work  to  be 
completed  more  accurately  and  faster.  Generally,  those  who  can  adapt  and  be  trained 
to  deal  with  this  new  technology  will  prosper.  Those  who  cannot  adapt  and  be  trained 
will  fall  behind  and  become  information  deprived  (Jonassen,  1984).  Likewise,  those 
teachers  who  are  not  trained  in  this  new  technology  will  not  only  fall  behind  and 
become  information  deprived,  but  their  students  will  also  be  affected. 

Power  on!  New  Tools  for  Teaching  and  Learning,  prepared  by  the  United 
States  Congress  Office  of  Technology  Assessment,  purports  that  there  is  a  willingness 
on  the  part  of  school  districts,  schools,  teachers,  and  parents  to  explore  the  use  of  new 
learning  technologies.  Growth  in  use  has  been  impressive  and  indications  suggest 
that  the  future  use  of  technology  remains  strong.  However,  present  conditions  still 
indicate  that  (1)  the  majority  of  schools  do  not  have  sufficient  hardware,  (2)  students 
spend  less  than  one  hour  per  week  using  a  computer,  (3)  poor  and  minority  students 
have  less  access  to  technology,  and  (4)  the  vast  majority  of  teachers  have  little  or  no 
training  to  use  this  technology  (OTA,  1988). 

Part  of  the  focus  on  educational  improvement  lies  with  teacher  training.  It  is 
the  teacher  who  makes  one  of  the  critical  decisions  about  instruction.  Schools  may 
change  in  the  future,  but  it  is  how  teachers  view  technology  and  their  "preparation  to  use 
technology  to  instruct,  manage  and  evaluate  are  critical  to  the  better  use  of  technology 
in  the  curriculum"  (Glenn  and  Garner,  1989). 

At  the  university  level,  schools  of  education  are  currently  being  required  to 
revise  their  preservice  teacher  education  programs  and  include  training  in  the  use  of 
technology  in  the  classroom.  There  are  approximately  1300  institutions  in  the  United 
States  that  educate  teachers  and  prepare  them  for  licensure.  While  California  has 
legislated  that  all  teachers  must  complete  a  technology  training  component,  other  states 
are  moving  toward  mandating  technology  training  as  part  of  the  licensure  process  for 
beginning  teachers  (Glenn  and  Carrier,  1989).  A  1979  Association  of  Educational 
Communications  and  Technology  (1982)  survey  disclosed  that  36  percent  of  the  states 
had  no  media  requirements  in  teacher  education  programs,  while  64  percent  of  the 
states  required  limited  media  instruction.  There  were,  however,  four  states, 
Louisiana,  Ohio,  Oregon,  and  Wyoming,  that  required  media  standards.  In  a  1989 
survey,  18  states  and  the  District  of  Columbia,  required  that  all  students  in  their 
teaching  degree  programs  or  those  seeking  certification  take  a  course  on  computer 
topics  or  demonstrate  familiarity  in  using  tech  nology  for  instruction.  Yet  half  of  the 
states  neither  require  nor  recommend  technology  preparation  for  new  teachers  (Fulton, 
1989). 

A  survey  conducted  by  the  American  Association  of  Colleges  for  Teacher 
Education  asked  faculty  and  students  in  90  member  institutions  to  evaluate  the 
effectiveness  of  their  teacher  education  programs  in  preparing  classroom  teachers. 
For  most  aspects  of  teaching,  e.g.,  planning  instruction,  evaluating  students' 
learning,  using  proper  teaching  methods,  etc.,  the  report  was  positive.  More  than  two- 
thirds  of  education  faculty  and  students  considered  graduates  to  be  prepared  to  assume 
tasks  of  classroom  teaching.  In  the  preparation  of  using  technology,  however,  the 
results  were  not  as  positive.  Fewer  than  one-third  of  the  students  reported  they  thought 
they  were  prepared  to  teach  using  technology  (Fulton,  1989). 

There  is  a  need  to  train  teachers  in  the  selection  and  utilization  of  instructions 
media  in  powerful  ways  to  significantly  enhance  learning  and  thinking.  The 
effective  selection  and  use  of  instructional  media  in  the  classroom  is  a  way  to  improve 
content  mastery  and  student  thinking  skills.  It  also  increases  individualized 


instructional  opportunities,  improves  student  attitudes  towards  learning,  and  it 
prepares  students  for  a  technology-oriented  society.  Instructional  media  also  can 
increase  teachers'  job  justification  and  increase  the  cost-effectiveness  of  instruction 
(Boe,  1989). 

Significance  of  Study 

The  assessment  of  attitudes  of  trained  and  untrained  preservice  and  full-time 
teachers  toward  the  selection  and  use  of  instructional  media  has  been  researched  since 
1959  (Kelly,  1959;  Cole,  1964;  Acquino,  1968;  Wheller,  1969;  Lasher,  1971;  Jenkins,  1972; 
Welch,  1974;  Long,  1974;  Kreamer,  1978;  Jones,  1982).  The  results  of  these  studies  have 
produced  a  mixture  of  findings.  Kelly,  Cole,  Wheller,  Jenkins  and  Long  found  that 
preservice  teachers  were  positively  influenced  toward  the  selection  and  use  of 
instructional  media  upon  the  completion  of  formal  training.  It  was  reported  that 
formal  training  produced  the  necessary  intellectual  and  motor  skills  to  create  a 
positive  attitude  toward  the  selection  and  utilization  of  instructional  media.  However, 
Acquino  found  that  formal  training  in  instructional  media  produced  negative  results, 
while  Lasher,  Kreamer,  and,  Jones  concluded  that  formal  training  did  not 
significantly  change  the  attitude  of  the  preservice  teachers  or  full-time  teachers  toward 
the  selection  and  utilization  of  instn  ^tional  media.  The  researchers  further 
concluded  that  there  was  no  relationship  between  attitude  and  the  amount  of 
professional  training.  Welch  concluded  that  the  attitude  of  the  teacher  was  not  a  major 
factor  in  determining  the  utilization  of  instructional  media. 

It  can  be  concluded  that  previous  studies  have  not  produced  definitive 
attitudinal  trends.  As  technologies  of  instruction  have  become  more  advanced  and 
sophisticated,  especially  with  the  advent  and  development  of  the  microcomputer,  there 
is  a  need  to  ascertain  and  describe  the  accumulated  knowledge  concerning  the 
attitudes  of  preservice  teachers  toward  the  selection  and  use  of  instructional  media  for 
several  reasons. 

First,  this  study  identified  and  verified  attitudes  associated  with  the  selection 
and  utilization  of  instructional  media  for  classroom  instruction.  Second,  this  study 
presented  a  report  of  preservice  .teacher  attitudes  toward  selected  instructional  media 
and  materials  which  can  be  reviewed  and  studied  by  other  researchers.  Third,  while 
research  related  to  this  study  was  conducted  in  the  late  50s,  60s  and  70s,  enough  time  has 
elapsed  and  the  learning  environment  in  schools  and  the  role  of  the  teacher  has 
changed  sufficiently  to  warrant  up-dating  current  preservice  attitudinal  trends  toward 
the  selection  and  use  of  instructional  media.  And  fourth,  this  study  provided  the 
assessment  of  preservice  teacher  attitudes  and  the  shift  in  attitudes  toward 
instructional  media  as  a  result  of  formal  training  and  student  teaching,  of  which  the 
established  body  of  attitudes  can  be  used  as  a  basis  for  further  attitudinal  research  and 
curricular  changes.  What  preservice  teachers  use  and  select  as  instructional  media 
when  planning  instruction  during  student  teaching  may  be  an  indicator  of  what  may 
be  selected  and  used  when  teaching  full-time. 

It  can  be  inferred  that  a  positive  attitude  toward  instructional  media  during 
formal  training  should  result  in  the  successful  demonstration  and  application  of 
instructional  media  when  teaching  full-time.  To  require  preservice  teachers  to 
receive  formal  training  in  the  selection  and  use  of  mstructional  media,  however,  may 
or  may  not  appear  to  be  the  independent  variable  that  will  determine  its  use  by  the 
teacher  in  the  classroom.  There  may  be  other  factors  that  have  an  influence  upon  the 
preservice  teacher's  decision  to  adopt  instructional  media  in  the  systematic  approach  to 
the  design  of  instruction.  Therefore,  how  preservice  teachers  rated  instructional 
media  and  materials  provided  additional  information  about  the  selection  and 
utilization  of  alternative  instructional  delivery  systems  used  in  the  classroom. 
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Statement  of  the  Problem 

The  nature  of  the  learning  environment  and  the  role  of  the  teacher  has  changed 
significantly  since  the  Kelly  study  and  other  studies  have  been  conducted. 
Improvements  and  changes  in  selected  technology  of  instruction  for  classroom  use 
have  occurred,  It  was  apparent  that  there  was  insufficient  current  research  that 
accurately  defined  the  present  status  of  preservice  teachers'  attitudes  or  changes  in 
attitudes  toward  the  selecting,  utilizing,  and  rating  of  instructional  media  after  formal 
training  or  after  student  teaching, 

Research  Questions 

Given  the  stated  problem  underlying  this  3tudy,  the  following  research 
questions  were  designed  to  address  the  problem: 

1.  What  are  the  attitudes  of  preservice  teachers  toward  the  selection  and  use  of 
instructional  media  prior  to  formal  training? 

2.  Are  there  positive  changes  in  attitudes  toward  the  selection  and  use  of 
instructional  media  in  the  classroom  after  having  received  formal  training? 

3.  What  are  the  attitudes  toward  the  selection  and  use  of  instructional  media 
prior  to  completing  student  teaching? 

4.  Are  there  positive  changes  in  attitudes  toward  the  use  of  instructional 
media  after  completing  student  teaching? 

5.  Are  there  differences  in  attitudes  toward  the  selection  and  use  of 
instructional  media  between  preservice  teachers  completing  formal  training  of 
instructional  media  and  preservice  teachers  completing  student  teaching? 

6.  Are  there  differences  in  attitudes  toward  the  selection  and  utilization  of 
instructional  media  between  those  preservice  teachers  prior  to  formal  training  and 
preservice  teachers  completing  their  student  teaching? 

7.  How  would  preservice  teachers  enrolled  in  a  formal  training  course  of 
instructional  media  and  preservice  teachers  completing  student  teaching  rate  specific 
types  of  instructional  media  and  materials? 

8.  To  what  extent  was  instructional  media  available  to  preservice  teachers 
completing  student  teaching? 

9.  To  what  extend  did  the  supervising  teacher  use  instructional  media  in  the 
classroom? 

10.  To  what  extent  was  instructional  media  used  by  preservice  teachers  during 
student  teaching? 

Hypotheses 

The  study  was  based  upon  the  following  hypotheses: 

Hoi:    Preservice  teachers  will  have  a  more  positive  attitude  toward  the 
selection  and  use  of  instructional  media  after  having  received  formal  training  in  the 
selection  and  use  of  instructional  media. 

Ho2:    Preservice  teachers  will  have  a  more  positive  attitude  toward  the  use  of 
instructional  media  after  having  completed  student  teaching. 

Ho3:     There  are  no  differences  in  attitudes  toward  the  use  of  instructional 
media  by  preservice  teachers  taking  a  formal  course  in  instructional  media  and 
preservice  teachers  completing  student  teaching. 

Ho4:     There  are  no  differences  in  attitudes  toward  the  selection  and  utilization 
of  instructional  media  by  preservice  teachers  prior  to  formal  training  and  preservice 
teachers  after  completing  student  teaching. 

Ho5:     There  are  no  differences  in  rating  of  media  types  and  materials  by 
preservice  teachers  enrolled  in  the  selection  and  utilization  of  instructional  media 
course  and  preservice  teachers  completing  student  teaching. 
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Survey  of  Literature 

Learning  psychologists  advocate  that  learning  should  first  come  from  direct 
experiences,  second  from  the  iconic  representation  of  films,  television  programs,  etc., 
and  third  from  the  symbolic  experience  of  textbooks,  lectures,  etc.  However,  the 
symbolic  representation  is  usually  the  first  employed  form  of  instruction  which  would 
include  reading  assignments,  class  discussions,  teacher-directed  presentations, 
question  and  answer  sessions,  etc.  However,  educational  technologists  would  advocate 
that,  ideally,  the  selection  and  utilization  of  instructional  media  should  be  employed  as 
the  learning  transitional  mechanism  between  the  direct  and  symbolic  experience  prior 

to  the  symbolic  stage. 

Previous  research  has  indicated  that  students  learn  from  the  use  of 
instructional  media  incorporated  into  a  well  written  lesson  plan.  Benefits  from  using 
instructional  media  include  a  vicarious  learning  experience,  precise  communication, 
increased  interest  in  learning  by  the  student,  and  increased  options  for  learning. 

When  conducting  research,  the  measurement  of  attitudes  is  difficult. 
Measuring  a  change  in  attitude  requires  using  even  a  more  complex  procedure. 
Attitudes  are  dynamic  due  to  the  amount  of  outside  stimuli  constantly  impacting  upon 
the  complexity  of  the  attitude  itself.  To  measure  an  attitude,  the  object  of  the  attitude 
must  have  a  precise  definition,  and  the  attitude  must  be  observable.  At  best,  the 
researcher  relies  upon  an  inference  of  the  attitude  since  it  is  impossible  to  directly 
measure  the  attitude  due  its  complexity.  It  is  also  important  for  the  researcher,  in  an 
attitudinal  survey,  to  measure  and  report  the  attitudes  of  the  group  and  not  the  attitudes 
held  by  the  individual  subject. 

To  further  complicate  attitudinal  research,  there  is  no  consensus  among  social 
scientists  as  to  the  definition  of  attitude.  For  this  research  study,  Mueller's  (1986) 
definition  was  adopted.  The  attitudinal  object  for  this  study  was  instructional  media. 

It  was  observed  that  several  previous  studies  indicated  that  a  positive  attitude 
toward  the  selection  and  utilization  of  instructional  media  would  result  in  increased 
use.  This  would  support  the  common  notion  that  a  positive  attitude  produces  a 
particular  behavior.  The  survey,  of  literature  disclosed  that  attitudes  do  not  determine 
particular  behaviors.  Attitudes,  however,  make  certain  behaviors  more  or  less 
probable  (Gagne,  1985).  Attitudes  can  be  described  as  response  tendencies  or  the 
readiness  to  respond.  The  researcher  cannot  predict  that  a  positive  attitude  toward  the 
selection  and  utilization  of  instructional  media  will  result  in  the  actual  selection  of 
instructional  media.  The  researcher  should  infer  that  there  is  a  greater  likelihood  that 
preservice  teachers  will  select  and  use  instructional  media  after  formal  training  and 
student  teaching  has  been  completed. 

The  survey  of  literature  found  that  positive  and  negative  attitudes  were 
produced  by  formal  training  in  the  selection  and  utilization  of  instructional  media. 
Kelly  (1959),  Cole  (1964),  Wheller  (1969),  Jenkins  (1972),  and  Long  (1976),  found  that 
preservice  teachers  were  positively  influenced  toward  the  selection  and  use  of 
instructional  media  upon  the  completion  of  formal  training.  Formal  training 
produced  the  necessary  intellectual  and  motor  skills  to  create  a  positive  attitude  toward 
the  selection  and  utilization  of  instructional  media.  However,  Aquino  (1968)  four  d 
that  formal  training  in  instructional  media  produced  negative  results,  while  Lasher 
(1971),  Kreamer  (1978),  and  Jones  (1982),  concluded  that  formal  training  did  not 
significantly  change  the  attitude  of  the  preservice  teachers  or  full-time  teachers  toward 
the  selection  and  utilization  of  instructional  media.  It  was  also  concluded  that  th^re 
was  no  relationship  between  attitude  and  the  amount  of  professional  training. 

The  literature  revealed  there  were  several  additional  variables  that  could 
influence  the  selection  and  use  of  instructional  media: 


o  5    AO  .* 


1.  The  presence  of  a  full-time  audiovisual  coordinator,  i.e.,  a  school  media 
specialist,  formally  trained  in  the  selection  and  utilization  of  instructional  media, 
would  create  a  positive  instructional  media  climate  (Miller,  1969). 

2.  There  is  a  positive  relationship  between  the  attitude  of  teachers  and  the 
support  received  from  the  administration  (Kelly,  1959,  Gray,  1971). 

3.  The  value  of  the  medium  of  instruction  does  not  necessarily  lie  within  the 
medium  itself,  but  how  it  is  perceived  by  the  teacher  and  students  (Dodge,  et  al.,  1974). 

4.  Teachers  are  affected  by  how  instructional  media  may  be  defined; 
traditional  or  innovative.  Teachers  held  more  positive  attitudes  toward  traditional 
media  than  innovative  media  (Bish,  1967). 

5.  Teachers  will  display  a  negative  attitude  toward  instructional  media  if 
they  believe  that  it  will  replace  them  in  the  classroom  (Everson,  et  al.,  1978). 

6.  Teachers  will  display  a  negative  attitude  toward  instructional  media  if  it  is 
described  as  technology  (Everson,  et  al.,  1978). 

7.  The  attitude  of  the  teacher  is  not  the  major  factor  in  determining  the 
selection  and  utilization  of  instructional  media.  An  innovative  curriculum  is  a  better 
predictor  of  the  selection  and  utilization  of  innovative  instructional  media;  a 
traditional  curriculum  is  a  better  predictor  of  the  selection  and  utilization  of 
traditional  media  (Seidman,  1986). 

8.  There  is  no  relationship  between  subject  area  taught  and  the  use  of 
instructional  media  (Kelly,  1959,  Jenkins,  1972). 

9.  There  is  no  relationship  between  grade  level  and  the  use  of  instructional 

media  (Jones,  1982). 

10.  The  attitude  of  the  teacher  toward  instructional  media  is  not  affected  by  a 
lapse  of  time  when  the  teacher  is  formally  trained  in  the  selection  and  use  of 
instructional  media  and  when  it  is  used  (Aquino,  1974). 

11.  There  is  a  positive  relationship  between  the  amount  of  teaching  experience, 
the  amount  of  media  vsed,  and  the  amount  of  professional  training  (Lasher,  1971). 

12.  Failure  to  deal  with  the  negative  attitude  that  instructional  media  will 
replace  the  teacher,  may  have  a  negative  affect  upon  the  teachers'  attitudes  (Everson,  et 
al.,  1978). 

Methodology 

This  quantitative,  descriptive  rtsearch  study  measured  the  attitudes  of 
preservice  teachers  toward  the  selection  and  use  of  instructional  media  at  two  levels  of 
teacher  training:  preservice  teachers  enrolled  in  formal  training  of  the  selection  and 
utilization  of  instructional  media,  and  preservice  teachers  completing  their  student 
teaching  requirement. 

The  nature  of  the  learning  environment  and  the  role  of  the  teacher  have 
changed  significantly  since  the  Kelly  study  and  other  studies  have  been  conducted. 
Improvements  and  changes  in  selected  technology  of  instruction  for  the  classroom  use 
have  occurred.  Insufficient  current  research  existed  that  accuratelv  defied  the 
present  status  of  preservice  teachers'  attitudes  or  changes  in  attitudes  toward  the 
selection,  utilization,  and  rating  of  instructional  media  after  formal  training  or  after 
student  teaching. 

Subjects 

This  study  was  limited  to  those  students  enrolled  in  the  preservice  teacher 
certification  course,  Introduction  to  Instructional  Media,  310:201,  at  The  University  of 
Toledo,  constituting  the  formal  training  group.  The  second  group  were  those  students 
completing  their  student  teaching  requirement  for  certification  during  Winter 
Quarter,  1991. 
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There  were  approximately  168  preservice  students  enrolled  in  the  Introduction 
to  Instructional  Media  course  and  approximately  170  student  teachers  completing  their 
student  teaching  requirement  during  the  1991  winter  quarter. 

Instrumentation 

Preassessment  Instrument  An  instrument  was  designed  to  obtain 
demographic  data  about  the  subjects  enrolled  in  the  formal  training  course, 
Introduction  to  Instructional  Media,  310:201.  The  background  information  obtained 
from  the  subjects  included  gender,  age,  the  grade  level  which  the  preservice  teacher 
would  teach,  grade  point  average  (GPA),  and  clas3  rank.  Part  1  of  this  self-reporting 
instrument  obtained  data  that  concerned  the  selection  of  media,  degree  of  skill,  and 
frequency  of  use.  Part  2  dealt  with  the  production  of  instructional  materials,  the  degree 
of  skill,  and  frequency  of  use.  Part  3  was  concerned  with  the  use  of  systematic 
instructional  design  techniques,  the  degree  of  skill,  and  frequency  of  use.  The  purpose 
of  this  data  was  to  increase  external  validity.  This  data  served  as  a  descriptive  profile 
of  the  subjects  involved  in  formal  training  of  instructional  media  for  this  study  (see 

Appendix  A).  . 

Attitude  Instrument.  A  survey  instrument  was  designed  to  ascertain  the 
attitudes  of  preservice  teachers  enrolled  in  formal  training  and  preservice  teachers 
completing  their  student  teaching. 

The  process  of  designing  the  attitude  survey  instrument  first  involved 
reviewing  existing  literature  that  contained  surveys  that  assessed  attitudes  toward  the 
use  of  instructional  media.  Several  attitude  surveys  were  reviewed,  including  Paul 
Dawson's  Media  Attitude  Profile  (Long,  1976),  and  a  fifty-seven  item  survey  and  the 
modified  version  of  this  Media  Attitude  Profile  (MAP)  survey  designed  by  Kreamer 
(1978).  A  third  attitude  survey,  designed  by  Chi-Yin  Ywen,  was  also  examined 
(DelFrate,  1987).  Second,  areas  of  research  interest  were  established.  These  areas 
included:  (1)  the  contribution  that  media  can  make  to  instruction;  (2)  the  positive 
attributes  of  instructional  media;  (3)  the  future  use  of  instructional  media;  (4)  taking  a 
required  course  in  instructional  media;  (5)  present  perception  toward  teaching  with 
instructional  media,  (6)  the  conditions  of  using  instructional  media,  (7)  course- 
specific  perceptions,  and  (8)  perceptions  toward  systematic  lesson  planning 
techniques.  The  third  step  included  the  writing  of  specific  response  items.  When  the 
response  items  were  written,  the  guidelines  of  the  behavioral-tendency  response  items 
of  would  and  should,  were  taken  into  consideration.  Would  items  (Mueller,  1986) 
express  a  personal  behavior  intention  toward  the  attitudinal  object.  Should  items 
express  a  behavioral  preference  for  social  action.  As  a  general  rule,  "items  inquiring 
about  respondents'  actual  behaviors  should  be  omitted"  (Mueller).  Therefore,  the 
response  items  included  the  term  should. 

The  collection  process.  This  study  used  the  self-report  method.i.e.,  the  subjects 
were  given  an  attitude  survey  instrument  and  were  requested  to  report  their  own 
attitudes  concerning  each  of  the  response  items.  This  type  of  data  collection  procedure 
represents  the  most  direct  type  of  assessment  (Henerson,  Morris  and  Fitz-Gibbon, 

1987).  ,  _  ,         ,  . 

The  attitude  survey  instrument  was  reviewed  by  a  panel  of  judges  and  changes 
were  made  upon  recommendations.  The  survey  instrument  was  tested  during  Fall 
Quarter,  1990  with  subjects  enrolled  in  the  Introduction  to  Instructional  Media  course, 
310:201.  Further  modifications  were  made  based  upon  the  results  obtained 

Semantic  differential.  The  survey  of  literature  also  revealed  another  form  of 
media  evaluation  in  the  form  of  a  semantic  differential,  as  was  used  by  Elliott,  et  al., 
(1984).  The  purpose  of  using  this  semantic  differential  was  to  obtain  the  ratings  of 
various  types  of  media  and  materials  used  in  Module  1,  2  and  3  for  the  Introduction  to 
Instructional  Media  course  as  well  as  those  preservice  teachers  completing  student 
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teaching.  This  semantic  differential  offered  the  respondents  sets  of  bipolar  adjectives 
with  a  seven-point  scale  separating  each  set  (Bellamy,  et  al,  1978).  The  respondents 
were  asked  to  mark  the  scale  point  which  most  nearly  represented  their  attitudes  The 
mean  score  across  the  groups  of  respondents  provided  data  which  served  to  establish  a 
base  line  for  selection  and  use  trends  by  the  subjects.  Two  forms  of  the  semantic 
differential  were  administered.  The  first  instrument  was  administered  to  the 
students  enrolled  in  the  formal  training  group  which  contained  four  sets  of  bipolar 
adjectives.  The  second  form  of  the  semantic  differential  was  administered  to  the 
student  teachers.  This  instrument  had  an  additional  set  of  bipolar  adjectives  and 
obtained  data  concerning  whether  the  media  were  available  or  not  available  during 
student  teaching. 

Student  teacher  utilization  profile.  To  obtain  data  concerning  the  selection  and 
use  of  instructional  media  while  the  preservice  teachers  were  completing  the  student 
teaching,  a  student  teacher  media  utilization  profile  instrument  was  designed  in  a 
similar  format  as  that  used  by  Welch  (1974).  The  purpose  of  this  instrument  was  to 
establish  the  types  and  amount  of  media  that  were  used  by  the  preservice  teacher  when 
completing  student  teaching 

Method  of  analysis 

The  pretest  ■  posttest  nonequivalent  control  group  design  was  used  to  analyze 
the  attitude  survey  for  the  first  hypothesis  (Wiersma,  1986),  i.e.,  preservice  teachers 
have  more  positive  attitudes  toward  the  selection  and  use  of  instructional  media  after 
having  received  formal  training  in  the  selection  and  use  of  instructional  media  The 
same  procedure  was  used  to  analyze  the  second  hypothesis,  i.e.,  preservice  teachers 
have  more  positive  attitudes  toward  the  use  of  instructional  media  after  having 
completed  student  teaching.  A  polled  difference  in  gain  scores  from  the  pretest  and 
posttest  attitude  scores  was  measured  by  using  a  t  -test,  two  sample  dependent  means  at 
the  .05  level  of  significance  (Hinkle,  Wiersma,  and  Jurs,  1988),  where: 

Ho:  8  =  ul  -  u2  =  0  and  Ha:  8  =  ul  -  u2  *  0. 
This  statistical  procedure  determined  if  there  was  a  significant  difference  in  attitude 
toward  the  selection  and  use  of  instructional  media  from  the  beginning  of  the  formal 
training  course  to  the  end  of  the  formal  training.  Both  preservice  intact  groups 
received  the  attitude  survey. 

,  _AP°stest  nonequivalent  control  group  design  was  used  to  test  the  third,  fourth 
and  fifth  hypotheses:  (Ho3)  there  are  no  differences  in  attitudes  toward  the  use  of  ' 
instructional  media  by  preservice  teachers  taking  a  formal  course  in  instructional 
media  and  preservice  teachers  completing  student  teaching;  (Ho4  )  there  are  no 
differences  in  attitudes  toward  the  selection  and  utilization  of  instructional  media  by 
preservice  teachers  prior  to  formal  training  and  preservice  teachers  completing 
student  teaching;  and,  (Ho5  )  there  are  no  differences  in  ratings  of  media  types  and 
materials  by  preservice  teachers  enrolled  in  formal  training  and  preservice  teachers 
completing  their  student  teaching.  Pooled  difference  in  scores  were  measured  by 
using  a  Mest,  two  sample  case,  and  independent  means,  where: 
Ho:  ul  =  u2  and  Ha:  ul  *  u2. 

The  statistical  package  SPSS-X  was  used  to  calculate  the  results  (SPSS-X  User's  Guide 
1988),  1 

To  analyze  the  extent  to  which  instructional  media  were  available  to  preservice 
teachers  while  completing  student  teaching,  survey  data  was  compiled  from  the  media 
rating  instrument  and  the  central  tendency  was  determined. 

To  analyze  the  extent  to  which  instructional  media  were  used  by  preservice 
teachers  and  their  supervising  teachers  while  completing  student  teaching,  the  data 
was  obtained  from  the  Student  and  Teacher  Utilization  Profile  and  central  tendency 
was  determined.  The  statistical  package  SPSS-X  was  used  to  calculate  the  central 
tendency  results  for  both  sets  of  data. 


Summary,  Discussion,  Conclusion,  Implications, 
and  Recommendations  tor  Further  Study 
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This  study  measured  the  attitudes  of  preservice  teachers  toward  the  selection 
and  utilization  of  instructional  media  used  in  the  classroom.  Two  groups  of  preservice 
teachers  were  involved  in  the  study.  The  first  group  were  subjects  enrolled  in  a  formal 
training  course  in  the  selection  and  utilization  of  instructional  media.  The  second 
group  of  subjects  were  preservice  teachers  completing  student  teaching  and  had 
already  completed  a  formal  training  course  in  the  selection  and  utilization  of 

instructional  media.  . 

An  attitude  survey  was  developed  that  contained  eight  specific  categories  to 
measure  the  subjects'  degree  of  agreement  toward  the  selection  and  utilization  of 
instructional  media.  These  categories  included:  (1)  the  contribution  media  can  make 
to  instruction;  (2)  the  positive  attributes  of  instructional  media;  (3)  the  future  use  of 
instructional  media;  (4)  taking  a  required  course  in  instructional  media;  (5)  present 
perception  toward  teaching  with  instructional  media;  (6)  the  conditions  of  using 
instructional  media;  (7)  course  specific  perceptions;  and,  (8)  perceptions  toward 
systematic  lesson  planning  techniques. 

The  study  was  based  upon  five  hypotheses  which  were  supported  by  research 

qUeStl°Hoi:    Preservice  teachers  have  a  more  positive  attitude  toward  the  selection 
and  use  of  instructional  media  after  having  received  formal  training  in  the  selection 
and  use  of  instructional  media. 

Ho2:    Preservice  teachers  have  a  more  positive  attitude  toward  the  use  ot 
instructional  media  after  having  completed  student  teaching. 

Ho3:     There  are  no  difference^  in  attitudes  toward  the  use  of  instructional 
media  by  preservice  teachers  taking  a  formal  course  in  instructional  media  and 
preservice  teachers  completing  student  teaching. 

Ho4:     There  are  no  differences  in  attitudes  toward  the  selection  and  utilization 
of  instructional  media  by  preservice  teachers  prior  to  formal  training  and  preservice 
teachers  completing  student  teaching. 

Ho5:     There  are  no  differences  in  rating  of  media  types  and  materials  by 
preservice  teachers  completing  formal  training  of  instructional  media  and  preservice 
teachers  completing  student  teaching. 

This  study  also  provided  demographic  data  concerning  the  type  of  subject 
enrolled  in  formal  training  of  instructional  media.  The  data  disclosed  that  the 
majority  of  students  were  sophomores  (58.6%)  and  juniors  (22.8%)  with  the  remainder 
of  the  students  (18.7%)  were  freshmen,  seniors,  and  undergraduates  with  degrees.  The 
subjects  had  a  combined  mean  grade  point  average  of  2.83  on  a  4.0  scale.  The  majority 
of  students  reported  their  preferred  teaching  level  was  elementary  school  (40.6%)  and 
senior  high  school  (28.7%).  The  remainder  of  the  students  indicated  special  education 
(16.8%),  junior  high  school  (4.9%)  and  K-12  (2.8%). 

The  preassessment  survey  concerning  the  amount  of  skill  and  use  of 
instructional  media,  producing  instructional  materials,  and  let.on  planning 
techniques,  indicated  that  the  subjects  had  limited  or  no  skills.  The  frequency  of  use  of 
instructional  media,  instructional  materials,  and  lesson  planning  techniques  also 
disclosed  that  the  subjects'  use  ranged  from  occasionally  to  seldom  used.  Therefore, 
the  preassessment  profile  indicated  that  the  subjects  enrolled  in  formal  training  of 
instructional  media  began  the  course  with  no  skills  nor  having  used  the  selected 
media,  materials,  or  formal  lesson  planning  techniques. 
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Also  provided  in  this  study  was  the  amount  of  media  used  by  student  teachers 
and  their  supervising  teachers.  This  data  was  obtained  using  a  Sfr  dent  and  Teacher 
Media  Utilization  Profile. 

Discussion 

Hypothesis  1:  The  pretest-posttest  for  Hypothesis  1,  preservice  teachers  have  a 
more  positive  attitude  toward  the  selection  and  use  of  instructional  media  after  having 
received  formal  training,  revealed  21  significant  differences  and  four  (4) 
nonsignificant  differences.  The  hypothesis  was  retained. 

The  following  discussion  is  an  analysis  of  the  attitude  survey  by  objective 
category. 

Category  1:  The  contributions  of  instructional  media.  The  formal  training 
group  strongly  agreed  that  instructional  media  can  present  accurate  information,  can 
create  interest  and  options  for  learning,  and  provide  a  vicarious  learning  experience. 

Category  2:  The  positive  attributes  of  instructional  media.  The  subjects 
strongly  agreed  that  a  teacher  should  be  at  ease  using  instructional  media.  The 
subjects  agreed  that  instructional  media  is  just  another  way  to  teach  and  that 
instructional  media  was  as  good  as,  if  not  better  than,  traditional  forms  of  teaching. 
The  subjects  were  undecided  or  uncertain  that  nonprint  media,  such  as  film,  a 
television  lesson,  etc.,  can  provide  all  the  necessary  information  for  learning. 

Category  3:  The  future  use  of  instructional  media.  The  subjects  strongly 
agreed  that  when  they  are  teaching  they  intended  to  use  instructional  media.  The 
subjects  agreed  that  a  beginning  teacher  should  incorporate  instructional  media  into 
lesson  plans. 

Category  4:  Taking  a  required  course  in  instructional  media.  The  subjects 
strongly  agreed  that  a  teacher  should  know  how  to  use  a  variety  of  instructional  media 
and  agreed  that  a  teacher  should  be  required  to  take  a  course  to  learn  how  to  use 
instructional  media. 

Category  5:  Present  attitudes  toward  teaching  with  instructional  media.  The 
subjects  in  all  five  survey  items  had  a  significant  difference  at  the  .0001  level.  The 
subjects  agreed  that:  (1)  a  highly  motivated  teacher  uses  instructional  media  when 
teaching;  (2)  when  instructional  media  is  used  the  teacher  is  using  acceptable  teaching 
strategies;  (3)  instructional  media  is  a  strategy  employed  by  effective  teachers;  (4) 
using  instructional  media  increases  student  learning;  and  (5)  teachers  who  plan 
lessons  that  are  learner  centered  use  instructional  media. 

Category  6:  Conditions  of  using  instructional  media.  The  subjects  agreed  that 
a  teacher  should  use  instructional  media  whether  they  are  rewarded  or  not.  The 
subjects  were  undecided  as  to  whether  or  not  instructional  media  should  be  used  when 
needed,  otherwise,  traditional  methods  should  be  used.  There  was  also  no  significant 
difference  for  this  survey  item. 

Category  7:  Course  specific  attitudes.  The  subjects  strongly  agreed  to  all  items 
in  this  category  although  there  was  no  significant  difference  for  survey  item  1,  i.e.,  to 
be  effective,  a  teacher  needs  to  know  how  to  produce  instructional  media  properly.  The 
subjects  strongly  agreed  thai:  (1)  a  teacher  should  know  how  to  properly  use 
instructional  media;  i2)  effective  teachers  should  be  computer  literate;  and,  (3)  to  be 
effective,  a  teacher  ought  to  know  how  to  use  instructional  media  properly. 

Category  8*  Systematic  lesson  planning  techniques.  The  subjects  were  in 
agreement  to  both  items,  i.e.,  effective  teaching  requires  systematic  lesson  planning, 
and  systematic  lesson  planning  is  necessary  for  any  teaching. 

Hypothesis  2:  The  pretest-posttest  for  Hypothesis  2,  preservice  teachers  have  a 
more  positive  attitude  toward  the  use  of  instructional  media  after  having  completed 
student  teaching,  yielded  24  no  significant  differences  and  1  significant  difference. 
The  hypothesis  was  rejected.  The  attitudes  of  the  students,  once  having  been  formally 
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trained  in  the  principles  of  selection  and  utilization  of  instructional  media,  were 
maintained  after  student  teaching, 

Hypothesis  3*.  The  posttest  scores  concerning  Hypothesis  3,  there  are  no 
differences  in  attitudes  toward  the  use  of  instructional  media  by  preservice  teachers 
after  taking  a  formal  course  in  instructional  media  and  preservice  teachers 
completing  student  teaching,  revealed  21  no  significant  differences  and  4  significant 
differences.  The  hypothesis  was  retained. 

Hypothesis  4:  The  attitude  survey  prettest  for  the  formal  training  group  and  the 
attitude  survey  posttest  for  the  student  teachers,  for  hypothesis  4,  indicated  no 
differences  in  attitudes  toward  the  selection  and  utilization  of  instructional  media  by 
preservice  teachers  prior  to  formal  training  and  preservice  teachers  completing 
student  teaching,  revealed  17  significant  differences  in  attitude  and  8  no  significant 
differences.  The  hypothesis  was  rejected. 

Hypothesis  5:  There  are  no  differences  in  the  rating  of  media  types  and 
materials  by  preservice  teachers  enrolled  in  formal  training  of  instructional  media 
and  preservice  teachers  completing  student  teaching.  The  comparison  between  the 
rating  of  media  by  the  formal  training  group  and  student  teachers  in  hypothesis  5 
revealed  no  significant  differences.  The  hypothesis  was  retained. 

Conclusion 

From  the  discussed  results  of  this  study,  the  following  generalizations  can 
be  made  and  serve  as  the  conclusion. 

1.  This  study  adds  a  new  segment  to  the  literature  indicating  that 
undergraduates  entering  a  teacher  training  program  do  not  have  developed  skills  in 
the  selection  and  utilization  of  instructional  media. 

2.  This  study  refutes  previous  research  that  concluded  formal  training  in 
instructional  media  had  either  no  effect  or  a  negative  effect  on  Jie  attitudes  of  the 
subjects.  This  study  has  demonstrated  that  the  intervention  of  a  formal  course  in  the 
selection  and  utilization  of  instructional  media  can  function  as  a  primary  factor  in  the 
development  of  students'  attitudes  in  a  positive  direction. 

3.  Students  in  a  teacher  training  program  who  have  developed  a  positive 
attitude  toward  the  selection  and  utilization  of  instructional  media  after  formal 
training  will  complete  student  teaching  with  as  much  of  a  positive  attitude,  or  even 
stronger  positive  attitude. 

4.  Of  the  ten  media  rated,  this  study  indicates  the  most  frequently  used  media 
during  student  teaching  were  the  personal  computer,  overhead  projector,  and 
television. 

5.  Based  upon  all  the  results  of  this  study,  a  developed  positive  attitude  toward 
the  selection  and  utilization  of  instructional  media  is  related  to  the  positive  affect 
toward  the  practice  of  systematic  lesson  planning  by  teachers. 

6.  The  instrumentation  used  in  this  study  was  effective  in  assessing  the 
attitudes  of  the  subjects  toward  the  selection  and  utilization  of  instructional  media. 
Moreover,  the  rating  of  instructional  media  led  to  outcomes  that  were  different  than 
those  reported  in  some  of  the  reviewed  literature. 

Implications 

Given  the  results  of  this  study  concerning  the  attitudes  of  preservice 
teachers  toward  the  selection  and  utilization  of  instructional  media,  it  is  in  the 
judgment  of  the  author,  there  are  several  implications  that  may  be  offered  in  the  areas 
of  teacher  training,  and  at  both  the  state  and  national  levels. 

Teacher  training.  Preservice  teachers  who  Lave  acquired,  through  formal 
training,  a  positive  attitude  '  oward  the  selection  and  utilisation  of  instructional  media 
are  more  likely  to  use  instructional  media  in  their  classrooms  when  they  teach.  This 
finding  is  due  to  the  development  of  selection  and  utilization  skills  that  have  been 
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acquired  during  formal  training.  Conversely,  there  is  a  high  probability  that  those 
teachers  not  having  been  trained  in  the  selection  and  utilization  of  instructional  media 
may  possess  negative  attitudes  toward  its  selection  and  may  not  use  instructional 
;nedia  in  their  classroom.  This  lack  of  skill  will  negatively  affect  both  the  teacher  and 
the  students.  It  is  important  that  a  positive  attitude  toward  instructional  media  be 
formed  early  in  the  preservice  teacher  training  program.  This  will  promote  the 
selection  and  utilization  of  instructional  media  when  teaching  full  time.  A  formal 
training  course  in  the  principles  of  selection  and  utilization  of  instructional  media 
can  shift  the  attitude  of  preservice  teachers  in  a  more  positive  direction.  Formal 
training  may  be  a  factor  in  shaping  attitudes. 

The  instructional  design  of  course  content  and  the  presentation  of  course 
content  in  a  formal  training  course  in  the  selection  and  utilization  of  instructional 
media  should  be  evaluated  to  determine  the  attitudinal  effect  upon  preservice  teachers. 

It  appears  that  once  a  positive  attitude  toward  the  selection  and  utilization  of 
instructional  media  has  been  formed  during  a  formal  training  experience,  this 
positive  attitude  will  be  maintained  throughout  student  teaching.  The  lapse  of  time 
between  formal  training  and  student  teaching  does  not  alter  the  attitude  but  reinforces 
the  positive  attitude. 

If  formal  training  is  a  factor  in  shaping  a  positive  attitude  toward  the 
selection  and  utilization  of  instructional  media,  it  would  be  appropriate  to  have  all 
faculty  and  administrators  who  are  involved  in  the  teacher  training  program  to 
receive  similar  training  and  for  faculty  to  be  encouraged  to  use  instructional  media  in 
their  teaching.  This  would  reinforce  the  contribution  of  media  to  instruction  and 
would  also  provide  positive  role  models  for  students  who  are  enrolled  in  the  teacher 
training  program  to  emulate. 

State  level.  The  findings  of  this  study  should  be  presented  at  the  state  level 
to  the  appropriate  educational  bodies,  such  as  the  State  Department  of  Education  or  the 
Board  of  Regents.  Policy  makers  at  that  level  should  consider  adopting  a  standardized 
formal  course  of  study  in  the  selection  and  utilization  of  instructional  media.  In 
addition,  the  decision  makers  should  be  encouraged  to  financially  support  such  an 
adoption  by  allocating  funds  to  equip  the  teacher  training  institutions  with  the  software 
and  hardware  necessary  for  proper  instruction.  Furthermore,  the  policy  leaders 
should  be  encouraged  to  allocate  funds  for  school  districts  to  purchase  needed  software 
and  hardware  that  will  enable  the  proper  selection  and  utilization  of  mediated 
instruction  in  the  classroom. 

National  level.  The  findings  of  this  study  need  to  be  shared  with  various 
professional  organizations,  such  as  The  Association  of  Educational  Communication 
and  Technology  (AECT),  and  The  Association  for  Supervision  and  Curriculum 
Development  (ASCD),  etc.  In  addition  to  formal  presentations  offered  to  the 
constituency,  the  promotion  for  and  the  adoption  of  formal  course  offerings  in  the 
selection  and  utilization  of  instructional  media  at  all  teacher  training  programs 
should  be  discussed  and  considered. 
Recommendations  for  Further  Study 

Based  on  the  analysis  of  the  results  of  this  study,  the  following 
recommendations  are  proposed  for  future  research: 

1.  This  study  should  be  replicated,  with  larger  samples,  in  other 
preservice  teacher  training  programs  to  determine  the  attitudes  of  the  preservice 
teacher  toward  the  selection  and  utilization  of  instructional  media. 

2.  A  longitudinal  study  should  be  conducted,  using  the  same  or  similar 
instruments  contained  in  this  study,  of  teachers  who  have  previously  received  formal 
training  in  instructional  media  and  who  are  in  their  third  to  fifth  year  of  teaching,  to 
determine  their  attitudes  toward  the  selection  and  utilization  of  instructional  media, 

3.  States  that  do  not  require  formal  training  in  the  selection  and 
utilization  of  instructional  media  should  replicate  this  study  with  various  preservice 
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teacher  training  programs  to  determine  the  present  attitude  of  preservice  teachers 
toward  the  selection  and  utilization  of  instructional  media. 

4.  A  study  should  be  conducted  to  determine  the  extent  to  which  media  are 
being  selected  and  utilized  by  teachers  who  have  been  formally  trained,  and  teachers 
who  have  not  been  formally  trained,  in  the  selection  and  utilization  of  instructional 

5.  Research  should  be  conducted  to  study  the  attitudes  of  experienced 
teachers  toward  instructional  media  and  determine  if  their  attitudes  are  related  to  the 
content  cf  their  teacher  preparation  programs. 

6.  Research  should  be  conducted  to  determine  if  there  is  a  relationship 
between  course  design  and  the  selection  and  utilization  of  instructional  media,  i.e., 
does  course  design  influence  attitudes  toward  the  selection  and  utilization  of 

instructional  media?  . 

7.  Studies  should  also  be  conducted  to  determine  if  there  is  a  relationship 
between  the  presentation  of  a  course  and  the  attitude  toward  the  selection  and  utilization 
of  instructional  media,  i.e.,  does  the  way  a  course  is  taught  affect  attitudes  toward  the 
selection  and  utilization  of  instructional  media? 

8.  Further  research  on  barriers  to  the  adoption  of  the  selection  and 
utilization  of  instructional  media  should  be  conducted. 

9.  A  study  should  be  conducted,  at  the  professional  level,  to  determine  the 
effects  of  inservice  training  on  those  teachers  who  have  previously  received  formal 
training  in  the  selection  and  utilization  of  instructional  media. 

In  summary,  course  experience  related  to  formal  training  in  the  selection 
and  utilization  of  instructional  media  does  produce  a  positive  attitude  toward  the  use  of 
technology  in  the  classroom.  Once  the  positive  attitude  has  been  developed  it  is  not 
altered  by  student  teaching. 
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EFFECTS  OF  LEARNING  STYLE  IN  A  HYPERMEDIA  INSTRUCTIONAL  SYSTEM 

by 

YUNG-BIN  BENJAMIN  LEE,  PH.D. 

Background 

The  development  of  hypermedia  technologies  has  led  to  a  great  anticipation 
of  applications  in  education.  A  hypermedia  system  can  (1)  support  fast  navigation 
and  linkage  of  related  information  for  associative  learning,  (2)  represent 
multimedia  elaborated  information  of  various  forms,  (3)  update  information 
easily,  and  (4)  provide  a  high  degree  of  freedom  for  learners  to  select  and  to 
sequence  information,  However,  current  interest  in  the  educational  possibilities  of 
hypermedia  has  focused  mainly  on  the  systems  implementation  and  software 
development,  Little  attention  has  been  paid  to  the  complexity  of  the  interaction  of 
design  strategies  and  learners'  characteristics  in  the  educational  applications  of 
these  capabilities, 

The  most  often  mentioned  capability  of  hypermedia  systems  is  the 
provision  of  a  high  level  of  freedom  for  learners  to  navigate,  to  select,  and  to  link 
related  information  of  different  forms,  To  accomplish  these  browsing,  selecting, 
and  linking  processes  in  order  to  view  related  information,  users  usually  have  to 
point  the  mouse  or  any  other  input  device  on  the  marked  text  or  a  graphical  icon 
called  button  and  click  on  it;  then  the  related  information  will  be  shown,  The 
underlying  assumption  is  that  learners  will  actively  choose  to  see  more  related 
information  to  form  a  knowledge  base  on  a  topic.  Additionally  learners  will  make 
use  of  the  related  information  in  conjunction  with  their  own  previous  knowledge  to 
enrich  their  knowledge  base,  This  assumption  requires  further  study,  When 
learning  in  a  hypermedia  system,  will  learners  actively  use  linking  capabilities 
to  seek  related  information  or  passively  accept  whatever  they  are  given?  If  learners 
act  differently,  what  factors  contribute  to  the  learning  style  variations?  Will  the 
frequency  of  interactivity  decrease  as  the  instructional  time  increases? 

Research  Questions 

The  study  was  conducted  to  answer  the  following  questions  regarding 
learning  in  hypermedia  instructional  systems, 

1,  Are  there  differences  among  subjects  of  different  learning  styles  on: 

a,  performance  tests  scores? 

b,  time  on  task? 

c,  selection  frequency  in  viewing  embedded  elaborated  information? 

d ,  attitudes  toward  the  hypermedia  system? 

e,  acceptance  of  instructional  advisement? 

2,  What  effect  does  instructional  strategy  have  on  subjects  of  different  learning 
styles  on: 

a,  performance  tests  scores? 

b,  time  on  task? 

c.  selection  frequency  in  viewing  embedded  elaborated  information? 

d.  attitudes  toward  the  hypermedia  system? 

3,  Are  there  interaction  effects  of  learning  style  and  instructional  strategy  on: 
a.  performance  tests  scores? 
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b.  time  on  task? 

c.  selection  frequency  in  viewing  embedded  elaborated  information? 

d.  attitudes  toward  the  hypermedia  system? 


Methods 


This  study  was  conducted  to  test  the  effect  of  learning  styles  and 
instructional  advisement  on  subjects'  achievement  test  performance,  frequency  in 
viewing  embedded  information,  time  on  task,  and  frequency  in  receiving 
instructional  advisement  in  a  hypermedia  instructional  system.  A  survey  on 
subjects'  attitudes  toward  the  instruction  and  the  hypermedia  system  was  conducted 
after  the  experiment  was  completed. 

This  was  a  two-session  study  with  three  weeks  interval  in  between 
sessions.  In  Session  One,  a  screening  test  on  DNA  and  protein  synthesis  was 
administered  to  participants  in  order  to  assess  their  knowledge  on  the  instructional 
topic  to  be  covered.  Participants  who  performed  80%  or  above  mastery  level  on  the 
topic  were  eliminated  from  the  subsequent  study.  A  learning  style  test  was  also 
administered  to  participants  in  the  first  session  of  the  study  in  order  to  determine 
the  group  assignment.  In  Session  Two,  subjects  were  taught  the  topic  using  a 
hypermedia  instructional  program;  then  an  achievement  test  was  administered.  A 
survey  of  their  attitudes  toward  hypermedia  and  the  subject  matter  also  was 
conducted  after  they  had  finished  the  achievement  test 

The  learning  styles  of  interest  in  this  study  were  (1)  passive,  (2) 
neutral, and  (3)  active  approaches  toward  learning  which  was  determined  by 
performance  on  the  Passive  Active  Learning  Scale  Test  (PALS),  a  32-item 
inventory  which  was  constructed  for  this  study  by  the  experimenter  to  measure 
learning  characteristics  of  different  learners. 

The  instructional  strategy  implemented  in  this  study  was  (1)  with  and  (2) 
without  advisement  at  the  end  of  each  instructional  segment  For  those  subjects  who 
received  the  advisory  version  treatment,  the  instructional  program  monitored  their 
interaction  with  the  subject  matter  along  the  instructional  segments  and  provided 
advisement  to  view  key  information  at  the  end  of  each  segment,  if  needed,  The 
other  treatment  did  not  provide  for  any  advisement,  and  served  as  control 
condition. 

The  experiment  was  a  posttestonly  control  group  design,  Two 
experimental  variables  were  arranged  in  a  3  x  2  factorial  design.  The  first 
independent  v?r;  ible,  type  of  learning  style,  consisted  of  (1)  active-learning  style, 
(2)  neutral-learning  style,  and  (3)  passive-learning  style.  The  second  independent 
variable,  instructional  strategy,  consisted  of  (1)  advisory  instructional  strategy 
(experimental)  and  (2)  nonadvisory  instructional  strategy  (control).  The  five 
categories  of  dependent  variables  were  (1)  performance  tests  scores,  (2)  time  on 
task,  (3)  selection  frequency  in  viewing  embedded  elaborated  information,  (4) 
attitudes  toward  the  hypermedia  system  and  the  instructional  material,  and  (5) 
frequency  of  receiving  instructional  advisement. 


The  results  of  this  study  indicated  that  achievement  test  scores,  time  on 
task,  and  selection  frequency  in  viewing  embedded  elaborated  information  were 
affected  by  the  interaction  of  learning  style  and  instructional  strategy  for  neutral- 
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learning  subjects.  Those  who  received  the  advisory  version  performed  better,  spent 
more  time  on  task,  and  chose  to  view  more  embedded  elaborated  information  than 
neutral-learning  subjects  who  received  the  nonadvisory  version.  Passive- 
learning  subjects  who  received  the  advisory  version  scored  significantly  higher  on 
their  achievement  test  than  passive-learning  subjects  who  had  the  nonadvisory 
version. 

The  results  of  comparisons  between  active-learning  and  passive-learning 
subjects  found  there  were  significant  group  differences  in  the  dependent  variables 
of  overall  time  on  task  and  selection  frequency  in  viewing  embedded  elaborated 
information,  with  active  learners  spending  more  time  on  task  and  choosing  to 
view  embedded  elaborated  information  more  often  than  passive  learners.  There 
was  a  trend  toward  a  difference  between  these  two  groups  in  achievement  test 
scores,  with  the  active-learning  group  scoring  higher. 

Educational  Implications 

The  results  of  this  study  indicated  that  learning  style  is  a  factor 
when  the  instruction  is  presented  with  a  hypermedia  system.  This  finding  suggests 
the  following  practical  applications  in  education: 

1.  Teachers  should  be  encouraged  to  assess  learning  stylos  of  their  students  in 
order  to  design  instructional  strategies  for  optimal  learning, 

2,  The  design  and  development  of  hypermedia  programs  should  consider  a 
student's  learning  style  and  be  flexible.  Instructional  programs  should  be 
easily  manipulated  for  presentation  of  material.  With  today's  technology  and 
computer  hardware,  these  capabilities  are  within  reach  for  educators  to  utilize 
for  their  students. 


510 


Title: 


Advancing  Distance  Education  Programs 
with  Ordinary  Technologies 


Authors: 

Randal  A.  Lemke 
Robert  C.  Loser 
Jeanne  L.  Manning 


541 


ADVANCING  DISTANCE  EDUCATION  PROGRAMS  WITH 
ORDINARY  TECHNOLOGIES 


The  use  of  ordinary,  relatively  low-cost,  and  readily  accessible  technologies  provides 
administrators,  designers  and  faculty  efficient  communication  tools  for  distance  education. 
Vo»ce  mail,  computer  conferencing,  audio  conferencing,  and  locally  produced  video  can 
enhance  the  quality  and  speed  the  rate  of  communication  for  distant  learners  and  faculty. 
The  latest,  exotic  and,  quite  often,  most  expensive  technologies  arc  the  focus  of  most 
conference  presentations,  professional  publications,  and  the  popular  press  as  they  represent 
future,  and  often  increased,  capabilities  for  distance  education.  Course  design  and 
administration  benefits  from  a  combination  of  both  kinds  of  technologies  and  this  paper 
focuses  on  the  use  of  ordinary  information  technoiogies. 

Communication  or  information  technology  has  been  at  the  center  of  discussion  of 
distance  education  from  the  earliest  correspondence  courses  to  the  prese  it  use  of  satellite 
communication.  Ease  of  use,  speed  and  access  are  crucial  in  the  design  and  administration 
of  instruction.  The  provision  of  access  to  courses  for  those  who  arc  not  served  by  campus- 
based  instruction  is  the  traditional  role  of  distance  education.  The  efficient  execution  of  this 
role  is  affected  by  the  speed  of  communication;  increasing  the  rate  of  transmission  has  been 
the  goal  of  many  in  distance  education  so  that  students  and  faculty  can  carry  on  a  dialogue 
about  the  course  subject  matter. 

Increasingly,  access  to  education  for  those  who  do  not  attend  classes  on  campus  can 
be  a  question  of  their  access  to  technology  The  higher  -  or  at  least  the  more  exotic  -  the 
technology,  the  fewer  the  students  who  have  a  means  to  use  it.  The  postal  service  provides 
access  for  almost  all  students  in  the  form  of  a  postal  address,  making  the  U.  S.  Mail  a 
universal  and  inexpensive  medium  of  communication.  It  is  a  medium  that  can  be  used 
without  traveling  from  home,  without  the  purchase  of  exotic  equipment,  and  without  the  need 
to  learn  how  to  use  new  technologies.  Students  have  access  and  know  how  to  use  this 
technology,  and  almost  all  forms  of  distance  education  use  printed  materials  relying  on  the 
postal  service  for  the  exchange  of  messages. 

Speed,  on  the  other  hand,  is  not  the  postal  service's  major  asset.  Two  weeks  is  a 
common  time  frame  for  the  submission  of  written  work,  its  grading,  and  the  return  of  it  with 
the  instructor's  comments.  Two  way  live  audio  and  video  television  communication,  with 
facsimile  or  computer  file  transfer  for  written  work,  is  obviously  a  more  immediate 
communication  system.  While  speed  is  its  asset,  its  liability  is  its  limited  access  because  it 
currently  requires  students  leaving  their  home,  the  procurement  of  televisions  and  other 
equipment,  and  user  training  for  staff  and  students.  Increasingly,  distance  education  units  are 
making  the  investment  in  the  fastest  and  latest  technology  which  gives  them  speed,  but  it 
sometimes  costs  them  accessibility.  While  innovation  in  the  use  of  the  newest  of  technologies 
should  continue,  a  parallel  move  to  use  existing  and  less  expensive  technologies  can 
complement  a  distance  education  unit's  communications  plan. 

This  paper  reports  on  Northern  Virginia  Community  College,  Extended  Learning 
Institute's  (ELI)  use  of  low-cost  technology  as  an  example  of  these  parallel  communication 
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plans.  In  particular,  it  examines  the  use  of  computer  conferencing,  voice  mail,  audio 
conferencing,  and  locally-produced  video.  With  a  base  of  85  courses  and  45  faculty,  ELI  has 
made  many  attempts  in  its  15  years  to  improve  communication  with  its  3000  students.  At 
present  it  uses  cable  and  broadcast  television,  audio  conferencing,  audio  and  video  tape,  print 
materials  sent  via  the  post  office,  computer  conferencing,  fax,  a  compressed  video  network 
connecting  its  five  campuses,  telephone,  and  voice  mail  to  communicate  with  students  on 
administrative  and  instructional  issues. 

COMPUTER  CONFERENCING 

Computer  conferencing  is  a  relatively  inexpensive  technology  and  student  access  to 
it  is  increasing.  Asynchronous  (not  real  time)  computer  conferencing  can  be  an  important 
distance  education  medium  in  that  it  provides  excellent  "broadcasting"  capabilities;  a  message 
typed  once  can  be  distributed  to  many  users  with  no  extra  effort.  Computer  conferencing 
makes  a  kind  of  "classroom  discussion"  possible  for  independent  honre  study  students,  a 
discussion  that  is  perhaps  more  thoughtful  and  more  open  than  in  the  classroom;  participants 
can  take  time  to  formulate  contributions  and  word  them  carefully,  and  slower  and  shyer 
students  have  more  of  an  opportunity  to  contribute.  Computer  conferencing  also  exercises 
students'  general  written  communication  skills. 

Computer  conferencing  can  be  an  expensive  and  inaccessible  medium  if  it  includes 
the  purchase  of  a  central  computer  with  hardwired  or  modem-based  terminals  for 
participants.  The  software  can  be  equally  expensive  and  require  much  staff  time  to  support. 
Finally,  it  can  also  require  much  time  and  effort  for  participants  to  learn  to  use  the  hardware 
and  software  and/or  time  and  effort  on  the  part  of  the  administrators  to  make  the 
conferencing  system  easy  to  use. 

At  ELI,  with  a  very  limited  budget,  we  looked  first  at  our  College's  existing  academic 
mainframe  computer.  This  computer  was  originally  intended  for  use  by  instructors  to  teach 
programming  and  systems  development  to  computer  science  students.  Some  use  is  still  made 
of  it  for  this  purpose,  but  the  migration  of  the  computer  industry  to  personal  computers  and 
local  area  networks  left  this  computer  underused.  A  student  programmer-oriented  operating 
system  (MUSIC)  with  a  rudimentary  electronic  mail  application  was  already  running  on  this 
computer.  In  addition,  there  were  hardwired  terminals  in  computer  labs  on  each  of  our  five 
campuses,  and  there  were  16  telephone  dial-up  ports  for  connection  by  modem,  All  of  the 
basic  hardware  and  software  needed  for  computer  conferencing  were  already  in  place  and 
being  supported  by  the  College's  computer  center.  What  made  this  technology  affordable  for 
ELI  was  that  it  was  in  place  and  the  computer  conferencing  system  could  ride  along  with  the 
College's  other  functions  including  existing  terminals  and  dial-in  ports. 

To  create  a  computer  conference,  ELI  staff  programmed  a  more  user-friendly 
interface  for  the  electronic  mail  software  and  a  mechanism  for  public  access  to  documents, 
created  student  and  instructor  accounts  that  defaulted  to  the  computer  conference  upon  log 
in,  and  wrote  a  detailed  user  manual.  All  of  this  was  done  simultaneously  with,  and  in 
support  of,  development  of  a  technical  writing  course  that  used  computer  conferencing  as  the 
primary  communication  medium.  The  proposal  for  this  course  by  an  instructor  was  the 
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driving  force  that  initiated  our  experiment  with  computer  conferencing.  Creation  of  the 
computer  conference  required  appro>uxitotfcty  two  months  of  staff  time  distributed  over  an 
eight-month  calendar  period. 

The  technical  wniinf  course  has  been  very  successful  over  the  past  two  years,  and  so 
has  a  psychology  course  beguii  a  y«ar  ago.  Students  enrolling  in  these  courses  receive  a 
course  guide  a  computer  confcwucnig  manual  in  the  mail.  The  course  guide  describes 
written  assignments  that  arc  m  be  ported  to  the  computer  conference.  The  manual  tells 
studc^lij  \\cav  to  conned  to  the  coittpcter  conference  and  how  to  write  and  post  an 
in! reduction  of  themselves.  Students  ore  encouraged  to  view  and  comment  on  each  other's 
r/jsignrnvrts,  wv!  to  work  collaboratively  when  interests  overlap.  The  instructors  provide 
public  v^oraonts  on  most  alignments,  but  give  grades  privately.  In  their  evaluations,  most 
stuckni*  say  that  ih«  cor&raicc.  Mty  lo  use  (but  limited  in  capabilities),  and  most 
appr  ^iate  llw  opjx>rtu: *r>  a*v  and  team  from  the  work  of  others. 

Studvus  have  iiie  ooiir,.<  of  3 : vhj ?  to  a  computer  lab  on  campus  and  using  a  terminal, 
or  of  using  their  own  persona!  computer,  modem,  and  telecommunications  software  to 
connect  to  the  computer  rwftrsnce  "via  the  telephone;  any  personal  computer  and 
telecommunications  software  that  cat;  emulate  a  VT100  terminal  can  be  used.  Many  of  the 
students  enrolling  in  the  course  use  personal  computers  at  home  or  at  work  to  do  their 
conferencing,  but  some  travel  to  the  campuses  to  use  the  terminals  in  the  computer  labs. 
Some  effort  was  required  to  communicate  to  computer  center  and  computer  lab  staff  the 
needs  and  problems  of  distance  learners  and  non-computer  science  majors.  Since  the  ELI 
staffs  idea  of  user-friendliness  differed  significantly  from  that  of  the  computer  center's  staff, 
it  was  sometimes  difficult  to  explain  why  certain  features  of  the  system  were  a  problem  and 
to  get  advice  on  how  to  circumvent  them.  There  were  frequent  problems  with  the  computer 
center's  dial-in  hardware,  which  had  probably  not  been  heavily  used  and  debugged  until  our 
project  began.  Computer  center  and  computer  lab  staff  were  oriented  toward  supporting 
daytime  campus  users  rather  than  nighttime  and  weekend  distance  users.  However,  the 
computer  center  was  quite  interested  in  seeing  the  academic  computer  better  used,  and  has 
made  an  effort  to  understand  and  support  the  needs  of  distance  education. 

In  choosing  to  use  a  relatively  old  mainframe  computer  as  the  conference  platform, 
ELI  elected  to  use  a  centrally  supported,  multi-modem,  mainframe  computer  with  a  line  editor 
instead  of  a  microcomputer  based  system  with  word  wrap,  block  editing,  and  automatic 
reformatting  functions.  While  the  mainframe  conference  has  its  limitations,  it  does  provide 
a  full-screen  format  and  the  capability  for  uploading  documents  students  create  on  their  own 
word  processors.  This  has  proven  sufficient  for  the  short  documents  required  of  students  in 
these  courses  and  is  an  excellent  means  of  communication  among  students  and  faculty. 

Current  ELI  plans  related  to  computer  conferencing  include  easing  our  administrative 
load  by  using  unenhanced  electronic  mail  software  and  student  accounts,  training  selected 
campus  instructors  to  use  computer  conferencing  to  support  classroom  instruction,  adding 
access  to  on-line  library  catalogs,  adding  computer-assisted  instruction  capabilities,  and 
providing  inexpensive  modems  through  the  College  bookstores.  All  of  these  will  be  done  with 
an  eye  toward  minimal  budgets  and  maximum  access  for  students. 
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Technology  and  electronic  media  have,  in  fact,  made  us  "distance  learners"  and 
changed  the  ways  in  which  we  can  communicate.  What  is  important  for  teachers  is  how  we 
use  this  technology  to  provide  learning  situations  which  are  comparable  to  traditional 
classroom  learning  situations.  This  is  particularly  true  for  the  foreign  language  teacher  and 
learner. 

In  the  Fall,  1990,  we  began  to  teach  French  through  a  distance  learning  television 
course.  Students  viewed  the  telecourse  French  in  Action  on  the  College  cable  TV  network 
and  sent  in  their  written  assignments  by  mail  and  fax,  their  oral  assignments  by  personal 
audio  cassettes.  The  provision  of  televised  lessons  was  relatively  inexpensive  in  that  the 
College  already  had  a  cable  tv  channel  and  access  was  open  to  all  who  had  cable  tv  or  who 
could  travel  to  one  of  the  College's  five  learning  resource  centers  to  view  the  lessons  on  tape. 

Immediately,  evaluations  from  students  as  well  as  concerns  from  the  foreign  language 
profession  made  us  look  at  more  effective  ways  to  deal  with  oral  assignments.  Students  told 
us,  that  while  they  appreciated  the  flexibility  of  the  TV  course,  what  they  missed  was  the 
interaction  between  teacher  and  student,  the  immediate  answers  to  questions  and  more 
important,  feedback  on  their  oral  production.  This  became  a  different  form  of  access  in  that 
it  required  a  faster  communication  method  that  what  could  be  done  via  the  mail.  Two  way 
television  capabilities  were  not  present  over  the  College's  cable  tv  channel. 

At  the  same  time,  the  foreign  language  profession  was  critical  of  distance  learning 
programs  because  students  in  these  programs  did  not  develop  adequate  speaking  skills.  The 
National  Council  of  State  Supervisors  of  Foreign  Languages,  together  with  the  American 
Council  on  the  Teaching  of  Foreign  Languages  have  developed  guidelines  (1990)  for  foreign 
language  distance  learning  programs.  These  are  to  include  live  interaction  between  student 
and  teacher,  activities  to  provide  for  listening  and  speaking  skills,  and  immediate 
individualized  feedback  on  student  oral  performance. 

The  voiw  mail  system,  with  a  personal  mailbox  for  the  instructor  and  a  personal 
mailbox  for  each  student  registered  in  the  course,  has  enabled  us  to  solve  this  access  and 
speed  problem.  The  system  was  originally  set  up  on  a  7-year  old  IBM  PC  compatible  with 
limited  memory  and  storage  capacity.  The  voice  mail  board  was  commercially  available  and 
required  some  knowledge  of  computers  to  write  the  files  necessary  to  set  up  the  board  and 
the  computer.  Currently  a  newer  computer  with  a  larger  hard  drive  and  more  voice  mail 
capabilities  (total  costs  about  $1000)  is  being  used. 

The  voice  mail  system  is  being  used  in  three  different  ways: 

1.  The  instructor's  personal  mailbox  serves  as  the  means  for  students  to  ask  questions 
and  leave  messages  for  the  instructor,  from  asking  about  a  difficult  grammatical 
point,  to  the  explanation  that  the  baby  has  been  sick  or  I  will  be  out  of  town  and  my 
assignments  will  be  a  little  late. 
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2.  Students  leave  their  weekly  dialogues  on  the  system  and  the  instructor  gives  them 
immediate  feedback  on  their  oral  production  on  their  personal  mail  box.  These 
dialogues  range  from  the  simple  10-line  dialogue  to  a  discussion  of  describing  a 
vacation  or  arguing  the  advantages  of  TV  publicity.  These  dialogues  are  designed 
to  simulate  telephone  conversations, 

3.  As  a  means  of  developing  listening  comprehension  skills,  each  week  there  is  a 
listening  comprehension  quiz  left  on  the  voice  mail  system  for  student?  Quizzes 
range  from  simple  recognition  of  vocabulaiy  and  grammar  from  the  text  to 
understanding  a  message  about  the  weather,  movie  schedules,  or  travel  arrangements. 
These  more  complicated  listening  quizzes  are  intended  to  simulate  daily,  real-life 
situations  of  calling  for  information  or  calling  to  talk  to  a  friend. 

The  advantages  of  the  voice  mail  system  have  been  many.  It  is  economical  to  install 
and  it  is  easy  for  students  to  use.  Everyone  has  access  to  a  telephone  and  students  are  used 
to  making  business  calls,  personal  calls,  and  calls  for  information.  The  system  is  flexible  and 
available  when  they  need  it.  Students  may  call  24  hours  a  day,  7  days  a  week  and,  when  their 
work  schedule  has  taken  them  out  of  town,  some  have  called  in  their  dialogues  from  out  of 
state. 

From  the  instructor's  point  of  view,  there  have  been  some  unforeseen  advantages. 
One  has  been  the  ability  of  the  voice  mail  system  to  personalize  the  relationship  between 
student  and  instructor  so  that  the  students  in  the  distance  learning  class  are  known  as  well 
as  students  in  traditional  classes.  Furthermore,  the  isolation  of  the  student  from  the 
classroom  has  been  helpful  io  a  number  of  students  by  lowering  the  anxiety  they  might  have 
experienced  in  speaking  in  front  of  an  instructor  and  classmates.  Several  of  them  have  said 
that  they  could  not  have  achieved  the  level  of  speaking  proficiency  had  they  been  in  a 
traditional  classroom  setting.  This  is  particularly  true  of  adult  learners  who  are  returning  to 
the  classroom  after  a  long  absence. 

AUDIO  CONFERENCING 

Audio  conferencing  is  an  inexpensive  means  of  allowing  students  and  faculty  to 
converse  without  the  need  for  anyone  to  commute  to  campus.  Audio  conferencing  is  a  fully 
developed  technology  and  one  that  the  State  of  Virginia  provides  through  a  "meet  me"  phone 
bridge.  At  a  prearranged  time,  students  and  faculty  call  a  designated  local  phone  number 
using  their  own  telephones  and  connect  with  each  other  in  a  multi-party  conversation.  A 
phone  bridge  provides  better  conversation  patterns  than  most  conference  call  systems  built 
into  students'  phones  since  more  people  can  participate  and  switching  between  speakers  is 
more  natural.  In  Virginia,  the  arrangement  of  these  conferences  is  extremely  easy;  it  takes 
only  telephone  call  to  a  central  operator  who  designates  a  phone  number  and  allots  time  to 
the  audio  conferees. 

Examination  reviews,  homework  help,  oral  practice  in  language  classes,  and  other 
instructional  applications  are  made  possible  with  the  phone  bridge.  Courses  that  are  paced 
for  group  work  can  especially  benefit  from  this  technology  because  it  allows  for  groups  to 
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meet  without  driving  to  a  campus.  It  does,  however,  require  that  everyone  participate  at  the 
same  time  since  this  is  a  real-time  telephone  conversations.  A  tape  delayed  version  of  this 
conference  can  be  made  available  for  class  members  who  do  not  participate  in  the  live 
conversation!  but  they  would  not  have  the  opportunity  to  contribute  to  the  discussion. 
Similarly,  since  this  is  an  audio  conference  no  visual  images  arc  shown,  but  they  can  be  sent 
in  advance  to  conference  participants  so  that  all  can  refer  to  the  same  documents.  Physical 
access  Ho  audio  conferencing  is  almost  total  as  most  people  have  telephones  and  the  speed  of 
communication  is  instantaneous.  Participation  can  be  limited  by  students'  and  faculty 
members1  busy  schedules. 

LOCALLY  PRODUCED  VIDEO 

Finally,  locally-produced  videos  can  be  used  to  capture  discussions  or  meetings  for 
show  later  time  to  other  class  members.  These  short  videc  programs  can  be  produced  by  the 
College  or  by  students  who  have  access  to  VHS  ca  orders.  Av  ELI,  the  College's  tv  studio 
offers  a  quick  means  to  capture  visual  images  with  accompanying  r,udio  and  distribute  them 
to  students.  Courses  that  rely  heavily  o\\  piinted  material  can  b^  enhanced  with  4-5  hours  of 
video.  These  4-5  hours  are  locally  produced  and  usually  frature  the  instructor  presenting 
visual  images  or  a  dialogue  that  wou? ■?  ;^ot  be  as  effective  in  print.  Production  is  simple  in 
that  the  College  has  a  telecourse  production  studio  that  uses  fixed  cameras  where  the 
instructors  switch  between  a  head  shot  of  themselves  and  a  graphics  camera  of  their  handouts. 
The  only  required  technician  monitors  the  recording  of  the  video  in  the  studio's  control  room. 
ELI  has  used  this  for  whole  courses,  for  the  4-5  hours  of  supplementary  video  mentioned 
earlier,  and  for  3  hours  of  wrap-around  video  that  accompanies  telecourses  licensed  from  PBS 
or  other  vendors. 

The  viewing  of  this  video  takes  place  on  cable  tv,  at  the  campus  learning  resource 
centers,  and  through  circulation  of  copies  of  the  tapes.  Production  of  video  by  students  will 
increase  as  they  n»ore  access  to  camcorders  and  begin  to  use  them  to  send  in  their 
assignments  or  othes  communications.  Speeches,  biology  laboratory  experiments,  history  field 
trips,  and  other  such  assignments  can  be  easily  recorded  on  a  camcorder  by  students  and 
submitted  to  instructors  for  review  and  feedback. 


Ordinary,  relatively  low-cost,  and  readily  accessible  technologies  are  prime  tools  for 
education  courses.  While  two-way  audio  and  video  may  be  a  primary  future 
technology,  telephones,  inexpensive  computer  connections,  and  locally  produced  video  are 
tools  that  can  be  employed  immediately  in  many  colleges  and  universities.  The 
communication  facilities  of  most  colleges,  as  in  the  case  of  NVCC,  ofTer  course 
administrators,  designers  and  faculty  readily  available  tools  for  distance  education.  Use  of 
these  in  voice  mail,  computer  conferencing,  audio  conferencing,  and  locally  produced  video 
can  speed  the  rate  of  communication,  increase  its  quality,  and  make  it  possible  for  faculty  to 
improve  the  learning  experiences  of  their  students. 
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The  Development  and  Evaluation  of  a  Self-Regulated  Learning 
Inventory  and  its  Implications  for  Instructor-Independent  Instruction 

Few  would  argue  with  the  claim  that  the  ideal  learner/student/scholar  is 
organized,  autonomous,  self-motivated,  self-monitoring,  self-instructing,  in  short, 
behaves  in  ways  designed  to  maximize  the  efficiency  and  productivity  of  the 
learning  process.  We  would  like  our  classrooms  to  be  filled  with  such  learners. 
Unfortunately,  the  rarely  are.  Our  students,  all  too  frequently,  are  underprepared 
and/or  unmotivated  with  respect  to  productive  academic  performance.  Given 
present  day  cultural  and  economic  conditions,  the  consequences  of  academic 
underachievement  can  be  disastrous,  both  for  the  individual  and  our  society  as  a 
whole  (National  Commission  on  Excellence  in  Education,  1983;  Jones  &  Idol, 
1990). 

Not  surprisingly,  a  general  call  has  gone  out  to  the  educational  community 
to  find  ways  of  improving  student  performance.  Such  improvement  will  require 
changes  on  a  variety  of  fronts.  Current  emphases  on  changing  our  curriculum, 
standards  of  achievement,  and  educational  choice  surely  represent  important  steps 
in  the  right  direction.  However,  equal  attention  must  be  paid  to  factors  more  or  less 
directly  under  the  learner's  control.  Too  great  an  emphasis  on  the  role  of  external 
conditions  and  factors  tends  to  suggest  that  student  performance  is,  in  large 
measure,  determined  by  forces  outside  of  the  learner's  control;  that  good  students 
are  the  products  of  education  rather  than  the  producers  of  educational  outcomes. 
Such  a  view  lacks  balance  and  may  be  seriously  misleading.  Why  are  some 
students  successful  despite  less  than  optimal  educational  conditions?  The  reasons 
are  surely  complex.  One  set  of  factors  that  are  likely  to  prove  significant  involves 
what  has  come  to  be  called  self-regulated  learning  (Borkowski,  et.  al.,  1990; 
Zimmerman,  1990). 

The  self-regulated  learning  perspective  is  multi-faceted  and  draws  on 
contemporary  developments  on  several  theoretical  fronts.  Nevertheless,  according 
to  Zimmerman  (1990 ,  p.4),  "a  common  conceptualization  of  these  students  has 
emerged  as  metacognitively,  motivationally  and  behaviorally  active  participants  in 
their  own  learning."  In  other  words,  self-regulated  learning  is  purposive,  goal 
oriented  and  involves  behaviors  designed  to  maximize  academic  performance. 
While  all  students,  barring  those  who  are  totally  tuned  out,  are  probably  active  in 
the  manner  just  described,  self-  regulated  learners  appear  to  be  both  more  keenly 
aware  of  the  relation  between  specific  behaviors  and  academic  success  and  more 
likely  to  systematically  and  appropriately  employ  such  behaviors.  Perhaps  most 
importantly,  self-regulated  learners  are  successful  learners  (Zimmerman  &  Pons, 
1986).  It  follows  that  understanding  the  behaviors  and  processes  that  underlie  self- 
regulated  learning  represents  an  important  goal  for  educational  researchers. 

Contemporary  approaches  to  self-regulated  learning  (although  not  limited 
to)  are  presently  dominated  by  two  main  theoretical  frameworks:  Social 
learning/cognitive  theory  and  information  processing  theory.  Zimmerman  and  Pons 
(1986),  for  example,  working  out  of  the  former,  have  defined  self-  regulated 
learners  in  terms  of  fourteen  dimensions  which  span  a  spectrum  from  cognitive  to 
behavioral  to  social  factors. 

A  variety  of  research  either  specifically  identified  as  focused  on  self- 
regulatory  processes  (Borkowski,  Carr,  Rellinger  &  Pressley,  1990;  Pressley  & 
Ghatala,  1990),  or  indirectly  concerned  with  self-regulatory  mechanisms  (Baker 
1989;  Brown  1978;  Glenberg,  Wilkinson  &  Epstein,  1982;  Justice  & 
Weaver-McDougall,  1989;  Leal  1987;  Spring,  1985),  has  been  conducted  from  an 
information  processing  orientation.  The  primary  dimensions  of  interest  for 
information  processing  theorists  include  metacognitive  processes,  learning 
strategies,  and  motivational  factors  related  to  self-attributions  (the  latter  also  being 
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an  area  of  interest  for  social-cognitive  theorists).  Clearly,  self-  regulated  learning  is 
a  multi-dimensional  and  complex  phenomenon  that  transcends  boundaries  of 
interest  that  have  separated  researchers  operating  out  of  differing,  and  sometimes 
competing,  theoretical  orientations. 

While  our  own  research,  at  this  juncture,  has  not  been  primarily 
theoretically  motivated,  we  did  find  it  necessary  to  impose  an  organizational 
structure  on  the  various  dimensions  of  self-regulated  learning  reported  in  the 
literature.  We  believe,  in  fact,  that  the  model  we  have  generated  may  be  one  of  the 
more  useful  elements  emerging  out  of  our  efforts.  Our  working  model  of  self- 
regulated  learning  presently  consists  of  five  dimensions:  Metacognition, 
learning  strategies,  motivation,  contextual  sensitivity  and 
environmental  utilization/control  (see  appendix  A  for  our  model  and 
examples  of  our  categorization  scheme).  In  developing  this  model,  we  reasoned 
that  the  successful  learner  must  both  internally  (we  use  the  terms  internal  and 
external  in  a  relative  sense  only  since  no  absolute  separation  in  the  meaning  of 
theses  concepts  as  psychological  constructs  is  possible)  regulate,  monitor,  evaluate 
and  modify,  when  necessary,  the  learning  process,  and  be  sensitive  to  and  utilize  or 
control  contextual  (external)  factors  such  as  course  and  instructor  demands,  where 
and  when  to  study,  who  to  go  to  for  assistance,  etc.  Most  of  the  various  self- 
regulated  strategies  reported  in  the  literature  fall  into  one  or  another  of  the  categories 
we  have  chosen.  We  argue,  for  example,  that  self-monitoring  and  self-evaluation 
are  best  construed  as  aspects  of  the  metacognitive  component  of  the  learning 
process  rather  than  as  independent  categories  as  in,  for  example,  the  scheme  of 
Zimmerman  and  Pons  (1986).  The  same  reasoning  can  be  applied  to  various 
categories  of  information  processing  reported  in  the  literature  (Pintrich,  Smith  & 
McKeachie,  1989;  Weinstein,  Zimmerman  &  Palmer,  1988)  which  we  subsume 
under  the  broad  notion  of  learning  strategies  (e.g.,  organizing  and  transforming, 
selecting  main  ideas,  restating,  etc.).  Similar  reasoning  led  us  to  subsume  the 
interesting  and  important  notion  of  epistemological  beliefs  (Shommer,  1990)  under 
the  general  category  of  motivation. 

Having  devised  a  model  of  its  components  we  felt  was  both  economical  and 
intuitively  compelling,  we  set  out  to  determine  if  self-regulated  learning  indeed 
played  a  vital  role  in  successful  academic  performance.  We  chose  to  do  this  by 
employing  a  self-report  inventory  of  our  own  design,  composed  of  five  subscales 
consistent  with  our  model  of  self  regulated  learning.  We  opted  to  develop  such  an 
instrument  because  (1)  to  our  knowledge  no  instrument  of  its  kind  existed,  and  (2) 
because  we  believe  that  measuring  the  extent  to  which  a  learner  is  self-regulating 
has  important  implications  for  designing  individualized  instructional  interventions 
not  typically  taken  into  account  by  instructional  designers.  In  what  follows,  we  first 
describe  the  development  of  our  diagnostic  tool  and  what  it  reveals  about  the  nature 
of  successful  academic  performance.  We  then  go  on  to  discuss  potential 
technology  based  applications  and  implications  of  our  work. 

Development  of  the  Self-Regulated  Learning  Inventory.  Our  first  step  in  the 
creation  of  a  self-regulated  learning  inventory  involved  the  generation  of  an  item 
pool.  We  decided  to  review  the  literature  and  to  construct  our  items  on  the  basis  of 
findings  that  reported  strong  relationships  between  learner-generated  activities  and 
academic  success.  The  result  was  a  pool  of  approximately  100  items.  We  then 
reviewed  and  analyzed  the  items  eliminating  those  that  were  too  much  alike  and 
rewriting  those  that  were  either  too  complex  or  too  vague.  This  left  us  with  a  pool 
of  seventy-one  items  all  of  which  were  included  in  our  first  instrument.  Although 
the  items  represented  five  subscales,  we  decided  to  present  them  randomly  as  a 
single  test.  A  five-point  Likert  scale  format  was  chosen  as  most  appropriate  for  this 
type  of  instrument. 
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A  pilot  run  was  conducted  to  see  if  directions  were  clear  and  sufficient,  how 
long  it  took  to  respond  to  the  inventory  and  if  the  items  as  written  were  clear  and 
comprehensible.  As  a  result,  a  formal  set  of  instructions  was  composed.  It  was 
determined  that  time  to  complete  the  inventory  ranged  from  20-30  minutes  (see 
appendix  B  for  sample  items). 

Subjects.  Our  subjects  were  all  students  enrolled  in  classes  in  the  college  of 
education  at  a  medium  size  mid-Western  University.  Unfortunately,  the  majority  of 
education  majors  continue  to  be  female.  Thus,  our  sample  contains  an  imbalance  in 
terms  of  males  (21)  and  females  (83);  a  weakness  we  are  seeking  to  redress. 
Permission  was  granted  to  administer  the  inventory  to  intact  classes  by  several 
instructors.  Classes  ranged  in  size  from  thirty  to  ten.  In  total,  the  inventory  was 
responded  to  by  120  students.  Only  104  cases  were  actually  analyzed  due  to  the 
failure  of  some  students  to  properly  report  requested  information  and,  in  some 
cases,  due  to  questionable  patterns  of  responding  such  as  circling  the  same  number 
for  every  (or  nearly  every)  item. 

Procedure.  The  inventories  were  administered  in  every  instance  by  either 
one  or  the  other  of  the  authors.  Having  obtained  prior  permission  from  class 
instructors,  we  passed  out  the  inventories  and  read  the  prepared  set  of  instructions. 
Although  participation  was  entirely  voluntary,  no  student  refused  to  fill  out  the 
inventory. 

Results.  We  first  report  on  findings  that  relate  to  the  technical  properties  of 
the  inventory.  Table  1  shows  the  results  of  an  analysis  of  internal  reliability  of  the 
inventory  and  its  subscales.  We  are  encouraged  by  these  results,  although  by  no 

Table  1 

Reliability  Coefficients 


MCS  LSS  MOS  CSS  ECS 


Alpha  .72  .',  .  .75  .59  .65 


means  satisfied.  Data  for  the  calculation  of  test-retest  reliability  were  unavailable  at 
the  time  of  writing  this  report,  hence  we  are  unable  to  provide  this  information  at 
this  time. 

An  analysis  of  the  correlation  between  each  of  the  items  on  the  inventory 
and  student  GPA,  as  well  as  a  correlation  of  each  subscale  item  with  the  total  score 
for  that  subscale,  was  conducted.  Three  items  failed  to  correlate  significantly  with 
either  subscale  total  score  or  GPA  and  will  have  to  be  replaced. 

Our  only  evidence  with  respect  to  validity  at  this  point  is  indirect.  That  is, 
our  items  were  constructed  on  the  basis  of  findings  in  the  literature  related  to  the 
construct  we  set  out  to  measure.  An  analysis  of  the  correlation  between  scores  on 
the  inventory  and  GPA,  our  measure  of  academic  achievement,  revealed  a 
significant  correlation  both  for  the  inventory  as  a  whole  and  for  each  of  the 
subscales  (see  Table  2).  This  result  corresponds  to  findings  as  reported  in  the 
supporting  literature. 


0 

ERIC 


5 


Table  2 

GPA  and  Scores  on  the  Inventory 


MCS       LSS        MOS       CSS        ECS  SRLTOT 
Correlation     .46**       .46**       .45**       .29*        .40**  .56** 
*p<.01,**p<.001 


As  noted,  we  selected  student  GPA  as  our  measure  of  academic 
achievement.  We  have  already  shown  that  scores  on  the  inventory  and  its  subscales 
correlate  significantly  with  GPA,  The  largest  coirelation  obtained  was  between 
GPA  and  total  score  (SRLITOT).  We  also  obtained  information  for  each  subject  on 
class  (F,S,  JR,  SR,  GRAD),  age,  sex  and  race.  Analysis  of  these  data  revealed  a 
significant  correlation  between  sex  and  total  score  on  the  inventory.  Table  3  shows 
the  scores  for  males  and  females  on  the  inventory  and  its  subscales.  It  can  be  seen 

Table  3 

Sex  X  Inventory  Score  (mean  scores) 


MCS  LSS        MOS  CSS  ECS  SRLTOT 

MALE**       57.6  61  50.7  34.25  35.4  238.95 

FEMALE*     57.49  66.4        54.3  36.44  38.01  252.95 

**  N=  83 
*N=21 


that  females  outscore  males  on  total  score  as  well  as  all  subscales  except 
metacognition.  While  these  differences  are,  in  most  instances,  statistically 
significant,  we  hesitate  in  drawing  a  firm  conclusion  due  to  the  small  number  of 
males  in  our  sample. 

The  data  for  scores  on  the  inventory  and  its  subscales  and  class  are 
presented  in  Table  4.  While  not  statistically  significant  (p<.  10),  they  do  indicate  the 
possibility  of  an  interesting  trend  in  student  development;  that  is,  that  students 
become  increasingly  self-regulated  as  learners  over  the  course  of  the  college 
experience. 


ERIC 


552 


520 


Table  4 

Class  X  Inventory  Score 


MCS  LSS         MOS  CSS  ECS  SRLTOT 


M 

SD 

M 

SD 

M 

SD 

M 

SD 

M 

SD 

M 

SO  55.8 

5.6 

63.5 

7.0 

52.4 

5.4 

35.2 

4.4 

36.9 

4.4 

243.8 

JR  57.8 

7.3 

67.2 

7.5 

53.7 

6.7 

36.8 

3.6 

37.8 

5.2 

253.4 

SR  58.4 

5.1 

64.8 

5.0 

54.7 

5.5 

35.4 

2.9 

37.4 

3.9 

250.9 

GR  61.0 

7.3 

65.3 

11.1 

54.5 

6.6 

38.5 

5.8 

39.5 

1.7 

258.8 

Total 
Possible 

85 

90 

75 

50 

55 

355 

Note:  SO 

-N=31,  JR- 

N=41,SR-N=27,  GR-N=5. 

Discussion.  Our  results  lead  us  to  conclude  both  that  self-regulated 
learning  is  an  important  component  in  academic  success  and  that  it  can  be  measured 
via  a  self-report  instrument.  The  results  of  our  analysis  of  the  data  indicate  a 
substantial  relationship  exists  between  self-regulated  learning  and  GPA.  This  result 
is  in  line  mm  published  research  on  self-  regulated  learning  (Zimmerman  &  Pons, 
1986, 1988;  Zimmerman,  1990).  The  fact  that  total  score  showed  the  highest 
correlation  with  performance  is  in  line  with  the  work  of  Zimmerman  &  Pons  (1 988) 
who  also  found  self-regulated  learning  could  be  treated  as  a  single,  overarching 
factor.  Our  results  further  suggest  that  successful  students  may  become  increasingly 
self-  regulating  over  the  course  of  the  college  experience.  This  finding,  however, 
needs  further  exploration. 

Our  research,  as  well  as  the  research  of  a  number  of  others  (Zimmerman, 
1990)  lends  support  to  the  claim  that  self-regulatory  skills  are  important 
components  of  successful  academic  performance.  The  question  is:  what 
implications  do  such  findings  have  for  the  design  of  instructor-independent 
facilitation  of  the  learning  process?  Two  elements  appear  to  be  essential  to 
designing  effective  ways  of  facilitating  the  acquisition  of  complex  cognitive  skills: 
an  understanding  of  the  learner  and  the  learning  process,  and  properly  anchored 
learning  contexts,  or  practice  environments.  Our  model  of  self-regulated  learning, 
we  think,  provides  a  viable,  comprehensive,  and  relatively  unique  basis  for  the 
former.  Our  inventory,  which  could  be  electronically  administered,  allows  for 
diagnostic  predetermination  of  the  level  of  self-regulation  present  in  a  particular 
learner.  Such  information  is  likely  to  prove  crucial  for  determining  the  degree  to 
which  a  particular  learner  is  ready  to  benefit  from  instructor-independent 
instruction.  The  fact  that  a  given  learner  may  be  more  or  less  able  to  self-regulate 
also  suggests  that  a  variety  of  instructional  options  must  be  developed  to  suit  the 
needs  of  different  types  of  learners.  Specifically,  it  may  be  necessary,  when  self- 
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regulatory  skills  are  deficient,  to  develop  instruction  designed  to  counteract  this 
deficiency.  As  to  the  problem  of  anchoring  instruction,  we  believe  that  computer, 
particularly  multi-media,  based  approaches  offer  the  most  viable  and  cost  effective 
solutions. 

There  are  several  reasons  why  we  believe  videodisc  technology  and  multi- 
media offers  a  viable  solution  path  to  instructional  interventions  of  the  kind  we  have 
in  mind.  As  noted,  a  general  theme  in  contemporary  instructional  psychology  is 
that  effective  instruction  must  be  contextually  grounded,  or  "situated"  (Brown, 
Collins  &  Duguid,  1989;  Cognition  and  technology  group  at  Vanderbilt,  1990). 
From  this  perspective,  instruction,  ideally,  would  take  place  within  real- world 
settings  resembling  as  much  as  possible  the  actual  contexts  in  which  the  skills  to  be 
learned  would  be  applied.  Practically  speaking,  however,  such  an  approach  is 
difficult,  perhaps  impossible,  to  effect  on  a  grand  scale.  To  make  matters  worse, 
such  a  strategy  presents  additional  difficulties  for  instruction  in  skills  considered  not 
part  of  the  standard  curriculum.  Videodisc  technology,  however,  lends  itself  to  a 
reasonable  compromise.  At  the  very  least  it  should  be  possible  to  create  problem 
contexts  that  resemble  closely  real-world  situations  for  students  to  ground  their 
learning  experiences  in.  In  terms  of  our  own  aims,  it  should  be  possible  to  recreate 
the  context  of  academic  tasks  faced  by  students  in  real  classrooms  by  using 
videodisc  technology  to  lend  a  sense  of  reality  to  instruction  aimed  at  promoting 
self-regulatory  skills.  At  the  same  time,  since  these  would  be  true-to-life 
simulations,  students  would  have  the  opportunity  to  try  out  various  strategies 
without  the  pain  of  a  failed  exam  or  course.  Furthermore,  since  metacognition  is 
vital  to  self-regulated  learning,  a  computer-based  approach  allowing  for  use  of  the 
instructional  program  as  a  temporary  self-reflective  "executive"  that  prompts 
(thereby  increasing  awareness),  monitors  and  evaluates  performance,  appears  a 
potent  method  for  the  building  of  this  crucial  component  of  skilled  learning. 
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Appendix  A 
Dimensions  of  Self-Regulated  Learning 
A  Working  Model 


A.  Metacognition 

Definitions 

1 .  Regulation  of  cognition 

a.  Planning/deciding 

b.  Monitoring 

c.  Evaluation/checking 

2 .  Knowledge  about  cognition 

a.  Knowing  what  to  do 

b.  Knowing  how  to  do 

c.  Knowing  when  to  do 

d.  Knowing  where  to  do 

3.  Self-reflective  awareness 


Types 

1 .  Metacomprehension 

a.  Text  processing 

b.  Listening  skills 

2.  Metamemory 

a.  General  strategy  knowledge 

b.  Metamemory  acquisition 
procedures 

c.  Specific  strategy  knowledge 


B.  Learning  Strategies 

Definitions 

1.  Plans  organized  to 
facilitate  successful  learning 

2.  Skills  specific  to  achieving 
learning  goals 

3 .  Procedures  that  accomplish 
academic  goals 


C.  Motivation 


Types 

1 .  Text  processing  strategies 

a.  Underlining  main  ideas 

b.  Summarization 

c.  Using  imagery 

2.  Lecture/discussion  processes 

a.  Notetaking 

b.  Graphic  representation 

c.  Recasting 


Definitions 

1 .  Awareness  of  the  relationship 
between  effort  and  outcome 

2 .  Sen  se  of  mastery/competence 

3.  Desire  to  learn 

D.  Contextual  awareness/sensitivity 


Types 

1.  Causal  attributions 

2.  Locus  of  control 

3.  Self-efficacy 

4.  Epistemological  Beliefs 


Definitions 

Ability  to  gauge  task  demands 
Ability  to  balance  task  demands 
with  personal  resources 
Ability  to  judge  the  relationship 
between  learning  task  and  assessment 


Types 

1 .  Cue  sensitivity 

2.  Congruence  assessment 


E.   Environment  utilization/control 

Definitions  Types 

1 .  Knowing  where  to  find  assistance  1 .  Help  seeking 

2.  Planning  and  scheduling  2.  Goal  setting 

3.  Establishing  a  learning  3.  Staging 
environment 
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Appendix  A  (continued) 

Self-Regulated  Learning  Model 
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Appendix  B 

Subscales  of  The  Self-Regulated  Learning  Inventory 
with  Sample  Items 


I.  Metacognition  Scale 

1 .  After  studying,  I  mentally  review  the  material  to  get  a  sense  of  how  much  I 
have  remembered. 

2.  When  reading  a  text,  or  reviewing  my  notes,  I  periodically  pause  and  ask 
myself:  Am  I  understanding  any  of  this? 

II.  Learning  Strategy  Scale 

1 .  When  preparing  for  an  essay-type  exam,  I  try  to  put  the  material  I  am 
studying  into  my  own  words. 

2.  When  I  need  to  remember  a  list  of  items  or  names,  I  actively  recite  or 
rehearse  them  until  I  can  recall  them  from  memory. 

III.  Motivation  Scale 

1 .  I  prefer  courses  that  are  moderately  challenging  to  easier  ones. 

2.  If  I  have  a  good  instructor,  I  do  well.  If  I  have  a  poor  instructor,  I  do 
poorly.  It's  that  simple. 

IV.  Contextual  Awareness/Sensitivity  Scale 

1 .  The  type  and  demands  of  a  particular  course  have  a  lot  to  do  with  the  kind 
and  amount  of  studying  I  engage  in. 

2 .  I  try  to  determine  what  a  particular  instructor  is  looking  for  in  terms  of 
performance  on  the  part  of  students  and  adapt  my  approach  to  the  course 
accordingly. 

V.  Environment  Utilization/Control  Scale 

1 .  When  I  study,  I  make  sure  I  have  enough  time  and  a  quiet  place  to  go. 

2.  If  I  find  I  do  not  understand  material  covered  in  a  text  or  in  a  course,  I  try  to 
get  help  from  someone  who  does. 
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AECT  Proceedings 
Abstract 

A  questionnaire  was  administered  to  one  hundred  and  sixty-seven  preservice 
elementary  education  undergraduates.  The  questionnaire  was  designed  to  assess  the 
relatedness  of  personological  variables  to  the  expected  use  of  computers  in  teaching 
by  preservice  student-teachers.  The  data  collected  comprise  the  baseline  for  a  live- 
year  longitudinal  study  that  is  to  be  conducted  over  four  phases. 

Discrepancy  Identified 

With  computers  becoming  increasingly  available  in  schools,  it  is  widely  believed 
that  educational  technology-especially  computer  technology -could  have  a  major 
positive  impact  on  improving  the  educational  system.  This  view  has  gained  the 
endorsement  of  federal  researchers  (see  The  National  Task  Force  on  Educational 
Technoloiiv  1986;  United  States  Office  of  Technology  Assessment  [OTA], 
1988)  .  academic  scholars  (Sheingold  and  Hadley,  1990). . .  and  educational  politicians 
(Albert  Shanker,  1990).  Perhaps  due  to  the  favorable  disposition  of  opinion  leaders 
towards  computer  technology  in  education,  computers  for  teaching  have  become 
increasingly  available.  In  the  period  from  1983  to  1987,  the  average  pupil-to- 
computerratio  in  public  elementary  schools  improved  from  112.4  to  36.8  pupils  per 
computer  (OTA,  1988).  . 

Yet,  in  spite  of  these  positive  trends,  computers  remain  underutilized  by 
teachers'  While  researchers  support  the  continuing  increase  in  the  availability  of 
computers,  they  note  this  caveat-that  the  key  to  profiting  from  computers  does  not 
he  in  their  sheer  presence;  but  rather,  in  the  way  in  which  they  are  used  (see  Berman 
&  McLaughlin,  1978;  NTFET,  1986;  Salomon  &  Gardner,  1986).  Unfortunately, 
a  relatively  small  number  of  teachers  have  integrated  computers  into  their  teaclung. 
Sheingold  and  118(116/3  survey  ( 1990)  has  shown  even  in  schools  where  the 
availability  of  computers  per  school  was  more  than  double  the  national  average  there 
was  only  about  one  teacher  per  school  who  integrated  the  computer  into  their 
teaching. 

Disturbingly,  there  is  a  strong  sentiment  among  teachers  that  several  promising 
breakthroughs  in  educational  technology  have  not  realized  their  widely-held 
expectations  for  revamping  education  (Ely  &  Plomp,  1988).  Note  the  assertions  of 
the  titles  to  these  articles  "Computer  Integration  in  Instruction  Is  Stuck.  (Dronka, 
1985)  or  .  "Computer  'Revolution'  [Is]  On  Hold. . . "  (O'Neill,  1990),  or, . . .  The 
Revolution  That  Fizzled"  (Bjerklie  &  Hollis,  1991).  Teachers  have  reported 
disillusionment  with  educational  technology.  Also,  some  writers  suggest  that  there  is 
little  research  evidence  on  the  effectiveness  of  educational  technology  and  that  which 
exists  provides  little  direction  for  teachers  in  how  to  use  the  technology  most 
appropriately  (Becker,  1987;  Roblyer,  1988).  The  opinion  of  and  use  of  educational 
computing  by  teachers  does  not  seem  to  agree  with  the  views  of  federal  researchers, 
academic  scholars,  and  educational  politicians  and  their  expectations  for  the  use  of 
educational  computing. 

We  hypothesize  that  there  may  be  several  personological  variables  which 
influence  teachers  to  use  computers  in  their  teaching.  We  reasoned  that  if  teachers 
still  did  not  integrate  computers  into  their  teaching  in  professional  educational 
enviromnents  where  computers  were  readily  or  reasonably  available,  then  an 
examination  of  selected  teachers'  internal    iriables  would  be  informative. 
Researchers  which  support  the  effect  of  personological  variables  on  behavior  include 
Gagne,  1978;  Gilbert,  1975;  Keller,  1983;  Vroom,  1966.  Before  discussing  the 
personological  variables  though,  the  situation  which  is  established  by  the  condition  of 
availability  is  considered. 
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Availabilitv/Expoaure/Cognitive  Dissonance 

The  availability  of  computers  to  teachers  in  a  school  causes  a  situation  that 
would  be  termed  "forced  exposure"  in  the  theory  of  cognitive  dissonance  (Festinger, 
1962,  p.  133).  This  means  that  exposure  to  the  computers  is  inevitable  and  occurs 
regardless  of  a  teacher's  intentions.  The  theory  predicts  that  in  this  situation  a 
person's  response  would  be  one  of  either  information  seeking  or  avoidance.  This 
situation  is  key  to  this  study  since  an  examination  of  the  personological  variables 
related  to  a  teacher's  response  to  computers  can  only  reasonably  occur  when 
computers  are  available  in  a  teacher's  work  environment.  Because  the  baseline  data 
were  collected  with  preservice  undergraduate  teachers,  their  responses  could  not 
reflect  their  actual  use  of  computers  for  teaching.  Rather,  the  preservice  teachers 
were  instructed  to  respond  to  the  portion  of  the  assessment  measuring  use  in  terms  of 
what  they  expec'ed  to  be  true  in  the  future  regarding  their  use  of  computers  in 
teaching.  In  the  later  phases  of  this  study  when  the  subjects  are  professional 
teachers,  the  availability  of  computers  in  the  subjects'  schools  will  be  required  in 
order  for  the  subjects'  to  remain  eligible.  Presently,  availability  has  been  defined  as  a 
maximum  pupil-computer  ratio  of  44  to  1  (Marcinkiewicz,  1991). 

The  availability  of  equipment  is  a  reasonable  prerequisite  for  the 
accomplishment  of  work,  but  availability  itself  is  insufficient.  One  line  of  thought 
suggests  that  forced  exposure  be  extended  to  forced  use.  In  other  words,  whether  or 
not  a  teacher  were  inclined  or  predisposed  to  use  computers  would  be  subordinated 
by  the  requirement  of  use.  A  realistic  expectation  is  that  teachers'  use  of  computers 
would  be  pro  forma  if  it  were  mandated  by  their  administration  (Berman  & 
McLaughlin,  1978,  p.  15).  What  is  not  clear  is  the  extent  to  which  teachers  would 
use  available  computers  if  they  were  not  directed  to  do  so.  The  following  section 
discusses  the  variables  which  will  be  examined  in  the  subsequent  phases  to 
determine  whether  they  are  predictive  of  teachers'  computer  use. 

Predictor  Variables 

The  variables  that  were  hypothesized  to  predict  computer  use  include  age, 
gender,  experience  with  computers,  innovativeness,  teacher  locus  of  control, 
perceived  self-competence  in  computer  use,  and  perceived  relevance  of  computers  to 
teaching.  The  first  three  demographic  variables  are  exogenous— they  are  influential 
but  are  not  subject  to  intervention.  The  latter  four  are  endogenous— they  are 
influential  and  are  subject  to  intervention.  The  selection  of  this  set  of  variables  is 
substantiated  by  well  founded  theories  and  by  empirical  evidence  which  relates  them 
to  computer  use.  Knowledge  of  the  predictive  relationship  of  any  of  these  variables  to 
computer  use  would  be  informative  for  intervention  with  teachers  or  for  differential 
staffing. 

Computer  Use 

Computer  use  was  operationally  denned  as  the  integrated  employment  of 
computers  in  the  classroom.  This  dimension  was  derived  from  a  model  of 
Instructional  Transformation  described  by  Rieber  and  Welliver  ( 1989)  and  Welliver 
(1990)  in  which  a  teacher  progresses  through  levels  of  involvement  with  computers. 
At  the  first  level,  a  teacher  becomes  familiar  with  computers  {familiarization);  then, 
begins  to  use  computers  in  teaching  (utilisation).  A  higher  level  of  use  is  observed 
when  the  teacher's  computer  use  becomes  critical  to  his  or  her  teaclung  (integmtion ). 
At  this  level,  teachers  consciously  and  inextricably  dele  gate  some  of  their  duties  to 
the  computer  and  are  aware  of  the  changes  to  their  role  as  a  result.  It  is  at  this  level 
too  that  the  teacher  is  aware  of  the  changes  to  the  teacher's  role  as  a  result  of  the 
integration  of  computers.  For  a  teacher,  this  new  awareness  of  the  restructuring  of 
instructional  activities  leads  to  a  pursuit  of  fine  tuning  the  computer-teacher-student 
relationship  (reorientation).  The  final  level  (evolution )  is  more  of  a  suggestion  than  a 
level -to  continue  practicing  and  learning  about  how  to  improve  instruction  through 
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the  systematic  implementation  of  computer  technology.  Key  to  this  level  is  that  the 
teacher  remain  sensitive  to,  be  prepared  for,  and  be  able  to  adapt  to  change.  An 
analogy  to  this  adoption  of  computer  use  would  be  that  if  upon  learning  to  drive 
(familiarization),  one  uses  a  car  to  improve  one's  general  transportation  in  order  to 
fulfill  a  variety  of  purposes  (utilization).  One  may  then  schedule  the  use  of  the  car  so 
that  the  vehicle  is  critical  to  the  fulfilling  of  the  purposes  (integration).  At  the  same 
time,  one  becomes  aware  that  the  new  technology  has  significantly  altered  aspects  of 
one's  role.  Computer  use,  therefore,  was  assessed  with  the  Levels  of  Use  (LU) 
assessment  (Marcinkiewicz,  1991)  which  had  been  developed  based  on  the  model  of 
Instructional  Transformation. 

Age 

The  relationship  of  age  to  computer  use  is  as  yet,  unclear.  Respondents  to 
Sheingold  and  Hadley's  survey  (1990)  of  outstanding  teacher-users  of  computers 
comprised  a  group,  over  half  of  whom  were  between  40  and  49.  In  research  on 
innovation  (Rogers  &  Shoemaker,  1971),  the  effect  of  age  has  shown  mixed  results. 
Rogers  (1983)  reports  that  the  more  mature  favor  change. 

Gender 

The  differences  in  computer  use  between  males  and  females  suggests  that  there 
may  not  be  equality  in  their  education.  Gilliland  (1990)  discusses  differences  in 
gender-attributable  perceptions.  For  example,  females  were  more  negative  in  their 
attitudes  towards  computers  and  expressed  lower  expectations  of  computer  use  and 
computer  usefulness  than  did  males. 

Sheingold  and  Hadley  (1990)  showed  an  equal  distribution  of  computer  users 
between  male  and  female  teachers.  Jorde-Eloom  and  Ford's  (1988)  findings  on 
gender  were  unclear  because  of  the  low  number  of  males,  although  being  male 
correlated  more  highly  with  several  variables  than  did  being  female.  Rogers  (1983) 
indicates  that  females  typically  showed  lower  levels  of  acceptance  of  innovative 
behavior  than  males.  Gender  differences  were  reported  to  correlate  significantly  with 
attitudes  towards  computer  use  (Dambrot,  Watkina-Malek,  Silling,  Marshall  & 
Garver,  1985).  Males  held  more  positive  attitudes  towards  computers  and  scored 
higher  in  computer  aptitude.  Goldman,  Kaplan,  and  Piatt  (1973)  found  that 
"mechanical  curiosity"  was  consistent  for  both  males  and  females  in  choosing  college 
majors  but  that  females  "needed"  less  of  this  attitude  than  men  even  if  they  were 
successful  science  majors.  Marcinkiewicz  (1991)  reports  that  gender  did  not  predict 
levels  of  computer  use  for  elementary  school  teachers. 

Experience  with  Computers 

The  amount  of  experience  an  individual  has  in  a  specified  behavior  contributes 
directly  to  the  individual's  general  tendency  of  expectations  about  that  behavior 
(Rotter,  1954).  Rogers'  (1962)  description  of  the  adoption  process  describes  an 
individual's  mental  interaction,  or  the  organization  and  elaboration  of  thoughts  and 
attitudes,  with  experience.  In  the  adoption  process,  progression  from  awareness, 
interest,  evaluation,  trial,  and  finally,  adoption  is  influenced  by  experience. 
Experience  is  also  elemental  to  Rieber  and  Welliver's  (1989)  description  of  the 
familiarization  level  of  the  model  of  Instructional  Transformation.  At  that  level 
familiarity  with  computers  develops  through  a  teacher's  experience  with  them. 

Computer  attitude  has  also  been  related  to  computer  experience  (  Dambrot  et  al, 
1985).  Jorde-Bloom  and  Ford  (1988)  found  that  experience  with  and  knowledge  of 
computers  and  experience  with  educational  innovations  were  among  the  factors  with 
the  strongest  positive  relationship  to  computer  use  by  early  childhood  education 
administrators.  For  Sheingold  and  Hadley's  survey  (1990),  73%  of  the  respondents 
used  computers  in  their  teaching  for  five  years  or  more.  Marcinkiewicz  (1991)  reports 
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that  computer  experience  did  not  predict  levels  of  computer  use  elementary  school 
teachers. 

Innovativeness 


Rogers  and  Shoemaker  (1971)  define  innovativeness  as  the  degree  to  wliich  an 
individual  is  relatively  earlier  in  adopting  innovations  with  respect  to  others  in  a 
social  system.  Innovativeness  is  a  stable  personality  measure  in  relation  to  the 
diffusion  of  innovation. 

In  their  development  of  the  Innovativeness  Scale,  Hurt,  Joseph,  and  Cook  (1977) 
have  focused  on  the  measurement  of  an  individual's  "willingness  to  change."  This  is 
the  focus  for  this  study.  An  individual's  predisposition  for  change  is  measured  by 
using  self-report  measurements.  Marcinkiewicz  (1991)  reports  that  innovativeness 
did  predict  levels  of  computer  use  by  elementary  school  teachers. 

Expectancy  Theory:  Valence.  Expectancy.  Instrumentality 

Three  variables  were  selected  to  represent  the  components  of  expectancy  theory. 
In  the  scheme  of  expectancy  theory  of  behavior,  there  are  three  components  of 
perception:  1)  valence,  which  refers  to  some  goal  one  values  or  desires;  2)  expectancy, 
which  refers  to  the  expectation  that  by  one's  effort  one  is  capable  of  achieving  some 
performance;  and  3)  instrumentality,  which  refers  to  the  belief  that  the  achieved 
performance  actually  results  in  acquiring  the  valued  goal  (Vroom,  1964).  For 
example,  expectancy  theory  would  predict  that  one  would  be  motivated  to  use  a 
computer  for  teaching  if  one  believed  that  expending  a  modest  amount  of  effort  would 
result  in  competent  use  of  the  computer  (expectancy),  and  using  the  computer  would 
be  instrumental  in  achieving  a  valued  goal  (instrumentality)  such  as  better 
instruction/learnini ,  The  goal  of  improved  instruction  is  valued  by  the  teacher 
(valence).  Valence  of  improved  learning  was  assumed  to  be  uniformly  true  for  this 
sample.  Expectancy  and  instrumentality  were  measured.  Expectancy  was  assessed 
by  teacher  locus  of  control  and  the  reported  subjective  probability  of  one's  capability 
to  use  computers  competently.  In  this  study,  computer  expectancy  referred  to  the 
degree  to  which  one  felt  capable  of  achieving  competence  in  using  the  computer  in 
teaching.  Instrumentality  was  gauged  by  assessing  one's  "perceived  relevance  of 
computers  to  teaching."  In  this  study,  this  refeired  to  the  degree  to  which  one 
perceived  computer  use  as  being  instrumental  in  or  key  to  achieving  the  valued  goal 
of  better  learning  by  students  (For  the  remainder  of  this  discussion  this  variable  will 
be  referred  to  as  perceived  relevance.) 

Locus  of  control  is  working  in  tandem  with  "self-efficacy'  theory  in  accounting 
for  the  expectancy  component  of  the  above  theoretical  framework.  Lor of  control 
refers  to  a  person's  belief  that  the  outcomes  of  a  behavior  are  dependt.    jn  one's  own 
actions,  fate,  or  luck.  Self-efficacy  refers  to  a  person's  belief  as  to  whether  or  not  he  or 
she  is  capable  of  a  certain  performance.  Studies  of  locus  of  control  have  supported 
prediction  of  a  person's  future  behavior  and  a  person's  initiative  in  controlling  his  or 
her  environment  (Rotter,  1966).  Studies  of  self-efficacy  have  predicted  performance 
not  predicted  by  locus  of  control  such  as  academic  performance,  tenden  cy  towards 
anxiety,  etc.  (Manning  &  Wright,  1983;  Taylor  &  Popma,  1990)  Vhe 
comprehensiveness  and  independence  of  the  two  constructs  as  measures  of 
expectancy  is  strongly  debated  (Bandura,  1982,  1989 ).  It  is  not  suggested  that  the 
measure  included  in  the  questionnaire  has  captured  the  complexity  of  the  construct 
of  s^lf-efficacy;  "the  subjective  probability  of  using  computers  competently  in 
teaching"  variable  may  be  related  to  self-efficacy  and  only  nominally  measure 
expectancy.  However,  by  including  this  measure,  a  different  aspect  of  expectancy  ran 
be  assessed  apart  from  that  assessed  by  locus  of  control.  (For  the  remainder  of  this 
discussion  uris  variable  will  be  referred  to  as  self  competence. ).  Gallo's  investigation 
(1986)  predicted  individuals  intentions  to  use  computer  technology  using  a  formula 
for  expectancy  theory.  Marcinkiewicz  (1991)  reported  that  "the  subjective  probability 
of  using  computers  competently  in  teaching"   a  concept  related  to  self-efficacy— did 
predict  elementary  school  teachers'  levels  of  computer  use.  Indeed,  in  Marcinkiewicz' 
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study,  teacher  locus  of  control  was  also  measured  but  was  not  identllied  as  a 
predictor  of  computer  use,  It  was  however,  correlated  with  the  self-competence 
variable. 

Measures  Selected:  the  Component  Assessments 

The  dependent  variable  in  this  study  was  three  categories  of  computer  use: 
1)  Nonuse;  2)  Utilization;  and  3)  Integration. 

The  independent  variables  were  1)  age,  2)  gender  3)  experience  with  computers; 
4)  perceived  relevance  of  computers  to  teaching;  5)  the  subjective  probability  of  the 
competent  use  of  computers  in  teaching,  6)  teacher  locus  of  control;  and  7) 
innovativeness. 

The  questionnaire  used  in  this  study  comprised  several  component  assessments 
(See  Table  1).  They  were  1)  the  Levels  of  Use  (LU)  assessment;  2)  the  Iruiovativoness 
Scale  (IS);  3)  the  assessment  of  perceived  relevance  of  computers  to  teaching;  4)  thn 
assessment  of  subjective  probability  of  capability  to  me  computers  competently  in 
teaching  (self-competence);  5)  the  Teacher  Locus  of  Control  s«  ale  (TLC)  and  0)  the 
Teacher  Role  Survey  (TRS).  Data  for  three  demographic  variables:  age,  gender,  and 
years  of  computer  experience  were  also  collected.  Also:  an  item  for  self-rating  one's 
use  of  computers  was  included  to  estimate  criterion-related  reliability  of  the  Lu 
assessment. 

Use  of  computers  was  assessed  with  the  LU  assoMsment  (Mareinkiewicz,  1991 ) 
in  which  paired  comparisons  were  used.  This  technique  was  appropriate  because  cut- 
off levels  of  computer  use  were  defined  which  were  mutually  exclusive  and 
exhaustive.  This  method  is  considered  the   . . most  exact  psychophysical  tool.  " 
useful  for  precise  information  concerning  judgments  or  pr**fi  Alices  (Nunnally,  1 959, 
p.  20-21).  The  levels  of  utilization  and  integration  are  represented  by  #o  items 
each.  One  item  from  the  utilization  level  is  paired  with  an  item  from  the  integration 
level.  The  response  procedure  is  forced  choice;  therefore,  the  subj  ct  is  directed  to 
select  the  statement  which  he  or  she  most  at  ongly  feeh  is  true  about  him  or  herself 
Because  the  subject  was  asked  to  respond  to  each  of  the  items  twice,  any 
inconsistency  in  responding  was  readily  evident. 

Validity  of  the  Levels  of  Use  Assessment  (LU) 

One  control  item  was  included  on  the  questionnaire  which  way  not  computed  in 
the  data  analysis.  Respondents  were  to  select  from  three  descriptions,  the  one  which 
best  matched  their  expectation  about  their  teaching  in  a  given  situation.  This  item 
was  written  to  reflect  the  same  dimension  of  use  as  in  the  LU  but  from  a  different 
perspective.  Its  function  was  to  provide  an  additional  measure  about  the  teachers' 
self-report  of  use  from  which  criterion-related  validity  could  be  established.  A 
measure  of  association  between  it  and  the  LU  was  computed,  subjects'  responses  to 
this  item  were  matched  with  their  levels  of  use.  Cohen's  kappa  (Crocker  &  Algina, 
1986,  p.  198-202;  Suen,  p.  1(55-167)  was  computed  to  estimate  the  consistency  of 
classification  of  the  measures  (kappa  -  .72). 

Reliability  of  the  Levels  of  Use  Assessment  (LU) 

Reliability  was  estimated  using  the  Coefficient  of  Reproducibility  (CR)  which 
also  demonstrated  that  the  items  of  the  LU  formed  ;/.n  onlered  scale.  The  (-P  for  the 
LU  component  of  the  questionnaire  was  Cli  ^  .96  (Mareinkiewicz,  U)91 ).  A  Ctt  of  fJ0 
is  considered  the  criterion  for  demonstrating  that  the  items  form  an  ordered  ;  cah  of 
allowable  response  patterns  (Bailey,  1987,  348;  Crocker  &  Algina.  Itfyti,  p  5ti) 
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N/A 

'/A 


J'Tmi;,  -lui-vph,  ;u.m  flc,  .!<'  Ml;  "Mniiov;  .    •  ms  N       l»  l'*vi>»petJ  '> 

capture  a  particular  a;5]-'  •  oii<;  -jvui  *  n,        wh  .   me;     o  uj  '»     >3]^  '-  ',i 

innovation  v/a«  a;  -aoyri;«ic  fis?*  tWu ■  .nt-a ■■.  ,   hicc  't  itit-i  ,  ^vd,  fjwiioli  a 

7~poiot  ucale.  Kwpnn  ''^  :«- ■■'  ^<>r<  * «,;  ;  t'c-i-^.  •  ,riv^f  ■  <r* 1  5  n.-u^j  «te!v 
ii-'3t!«;  4  -  luidech      ,;  :-  ui*  '  -r*U  iv-du.  .,  -»r       i-.i.t.i  vc«  ,  ■  ;iy  'i^^v 

'P     scorlsr^  liafci  be'.     I."  .^i-u  s<  "'-:tf,    bbv    •   "?iic  i  >  s  *  -  !■  *t'  -      a*i'h'    'S  '?f 
iimuvafciveneaa.  T^ftn1      :wo         ;  :uV:  ^    «...    -n.-  nnl       ■  >u  ft. if.  •nu 
nhoi-U-  -fid  KMt-.'in  v'fftm  .Tlu-  h^*-l^  t^c*  l>.>   -  ".v;t.  :^<Ii<    x\MU'..>  U    ';v!u^  -tiu-  * 
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the  questionnaire.  Its  use  was  supported  by  the  strength  of  its  relationship  to  the 
original  (r=  .92). 

Validity  of  the  IS 

The  authors  (Hurt,  Joseph,  and  Cook,  1977)  claim  of  construct  validity  is  based 
on  the  correlated  distribution  of  responses  of  thei  study  with  the  distribution  of 
innovator  types  identified  by  Rogers  and  Shoemaker  (1971)  on  the  basis  of  actual 
adoption.  Secondly,  innovativeness  is  characterized  as  being  unidimensional.  The 
factor  analyses  conducted  by  Hurt  Joseph,  and  Cook  revealed  that  the  factor 
structure  of  the  IS  was  nnidimensu»ri»l. 

Evidence  has  been  reported  supports  g  the  pi? iictive  validity  of  the  IS. 
vVitteman  (197U)  hypothesized  that  high  lev^la  of  innovativeness  would  be  related  to 
high  levels  of  opinion  le.-dership  Witteman  also  hypothesized  that  innovative  ties  a 
would  be  negatively  related  with  communication  apprehension.  There  was  a 
significant  linear  ■  rrelation  of  .50  between  the  responses  to  the  IS  and  the  measure 
of  opinion  leadership  (n  -  936).  There  was  a  significant  linear  correlation  .1  4b' 
between  responses  to  the  IS  and  the  measure  of  communication  apprehension. 

Reliability  of  tho  IS 

Reliability  for  the  20-item  form  of  the  instrument  was  estimated  using  a 
KR  -  20  for  maKing  all  possible  split-half  comparisons  (Hurt,  Joseph,  &  Cook,  L977 ). 
This  analysis  resulted  in  an  estimated  reliability  coefficient  oiKR-20  =  .94 
Furthermore,  all  of  the  20  items  "discriminated  significantly"  ( 1977 )  between  the 
upper  and  lower       of  the  distribution.  The  shortened  form  was  made  up  of  10  of 
the  original  20  items  whi"h  had  the  highest  item-total  correlations.  The  10-item 
shortened  form  has  an ".. .internal  reliability  of  r  =  .89,.. and  its  correlation  with  tho 
20-item  scale  is  r  =  .92. 

The  Perceived  Relevance  of  Computers 

Subjects  rat  ed  their  agreement  with  an  item  sla  ing  that  compute!  w  ire 
relevant  to  teaching.  They  rated  their  perceptions  on  a  7-point  scale.  KeuponsM.'  w^re 
scored  as  7  -  strongly  agree:  6  =  agree;  5  =  moderately  igr  v;  4  =  undecided; 
3  -  moderately  disagree;  2  -  disagree;  1  =  strongly  disagree 

The  .Subjective  Probability  of  ( "'<  '.mpetence 

Subjects  mted  their  agreement  witl  au  item  statin!;;  that  the ,  p-  reeived 
themselves  as  capable  of  using  the  comj  iter  competently  in  teaching  'hey  rated 
their  expectations  on  a  7-point  scale  f  an  strongly  agree  to  strongly  disagree  il). 

Teacher  Related  Locus  of  Control  Scales 

A  principle  question  i-<  this  study  was  to  determine  wne4to  ••  \  teacher's  locus  of 
euntooi  in  related  to  his  or  her  level  of  computer  use  .  The  Teaehu  Lo<  is  of  Control 
(Ti;  '•)  Scale  hv  Rose  and  Medway  and  the  le.mlu  i  'ole  Survey  (TRS)  scale  were 
selected  because  they  were  designed  to  measure  the  i  ui  jtrnct  for  teachers. 

Teacjie' •  Locu a.  o Pf  1< >utrol  Seal e 

The  TLC  was  appropriate  because  it  measui      •<  chers'  beliefs  in  their  control 
ever  atudent  outcome  This  is  the  same  construct  exprcw.  t.  '  h.  the  l^uid  report;  for  the 
T  ]  8  Dep-o-tmen;  of  Health.  Kdneatiou,  a  »d  Weliaie  (Herman,  &  McLaughlin 
i  !)V  8)  aa  being  the  most  instrumental  in  pre'.'!  'toy  implementation  of  innovai  an  loi 
to-.. -he.-  atti  Unites.  The  TRS  was  appropriate  beeauv*  't  focuses  on  teacher;  control 
ever  heir  sit  .atinn  aa  professionals 
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The  TLC  scale  comprises  28  forced-choice  items  where  respondents  must 
endorse  an  alternative  identifying  either  external  or  internal  control  of  classroom 
events.  Fourteen  of  the  items  describe  positive  or  success  situations  and  14  describe 
situations  which  are  negative  or  failure  situations.  Each  item  has  two  stems 
representing  the  alternate  loci  of  control.  Separate  scores  emerge  for  each  subscale: 
the  belief  in  internal  responsibility  for  student  success  (1+)  and  student  failure  (I-). 
Scoring  is  1  point  for  the  internal  response  only.  The  range  of  scores  possible  is  from 
0  to  28.  Since  the  partitioning  of  values  by  the  subscales  was  not  relevant  to  this 
study,  the  total  of  all  internal  responses  was  used  for  a  respondent's  locus  score. 

Validity  of  the  TLC. 

Rose  and  Medway  (1981)  report  two  validity  studies.  In  the  first  study  of 
criterion-related  validity,  the  TLC  and  Rotter's  (1966)  original  locus  of  control  scale, 
the  I-E  scale,  were  administered  simultaneously  to  teachers  undergoing  training  in 
innovative  educational  practices.  After  the  training,  the  teachers  were  rated  for  their 
implementation  of  those  practices  by  independent  raters.  Correlations  for  the  TLC 
scores  and  the  I-E  scores  with  the  independent  ratings  of  teachers'  implementation  of 
innovative  educational  practices  revealed  that  the  TLC  was  related  to  the 
independent  ratings  whereas  the  I-E  was  not.  The  correlations  are  reported  in  Table 
2.  The  1+  refers  to  the  subscale  measuring  internality  in  the  context  of  student 
success  situations.  The  I-  refers  to  the  subscale  measuring  internality  in  the  context 
of  student  failure  situations. 


Table  2 

CniTRlation  (r)  Values  for  the  TLC  Subscales  and  Rotter's  I-E  Scale 
with  Independent  Ratings  of  Teachers'  Implementation  of  Innovations 


1+  (success) 

I-  (failure) 

Rotters  I-E 

.no* 

.46* 

.28 

.46* 

.36 

19 

.34* 

.40* 

-.07 

.48* 

.42* 

.09 

.48* 

.44* 

.01 

.55** 

.62***. 

-.09 

.37 

-.18 

(*p  <  .05;  **p 

<  .01;  ***p  < 

.001) 

In  the  second  study  (Rose  and  Medway,  1981 ),  the  predictive  validity  of 
teachers'  locus  of  control  to  teachers'  classioom  behavior  types  was  examined  Both 
the  TLC  and  the  I-E  scales  were  administered  to  teachers.  For  six  types  of  behavior  . 
studied,  the  I-E  resulted  in  only  one  significant  correlation  while  there  were  five  for 
the  TLC. 

Reliability  of  the  TLC 

Internal  consistency  was  calculated  from  the  residts  of  the  final  trial  of  four  pilot 
hi  .^ins  comprising  89  elemf  ntary  teachers.  All  item-to-total  score  correlatiuns  were 
iiigmficnnt  (p  <  .01).  Kuder  Richardson  formula  20  reliabilities  were  .81  for  the 
T-  imbacaio  and  .71  for  the  I  r  subscale 
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Teacher  Role  Surve  v  (TRS) 

The  TRS  scale  (Maes  &  Anderson,  1985)  includes  24  forced-choice  items  for 
which  teachers  report  their  perceived  power  in  their  role  as  teachers.  Each  item  has 
two  stems  representing  the  alternate  loci  of  conrn>l.  By  design,  sealing  is  1  point  for 
the  external  response  only.  However,  the  scoring  was  reverse^  m  this  study;  the 
internal  scores  were  totaled  so  that  the  value  for  TRS  locus  of  control  would  be 
consistent  with  the  TLC  value  of  locus  of  control.  The  range  of  scores  possible  is  Ircm 
Oto  24. 

Validity  of  the  TRS 

Anderson  and  Maes  (1986)  report  criterion-related  validity  for  the  TRS  which  is 
supported  by  1)  a  significant  positive  correlation  (r  =  .31; p  <  .01)  between  the  TRS 
and  the  Rotter  I-E  scale  (Rotter,  1966);  2)  a  significant  negative  correlation -as 
predicted-(r  =  -.27;/)  <  .01)  between  the  TRS  and  the  Responsibility  for  Student 
Achievement  (RSA)  Total  (Guskey,  1981);  and  3)  a  significant  positive  con-elation 
between  the  TRS  and  four  symptoms  on  the  Symptom  Questionnaire  (Kel  .ier  & 
Sheffield,  1973):  anxiety-r  =  .31,  p  <  .005;  depression-r  =  .35,p  <  .001; 
hostility-r  =  .28,  p  <  .01;  and  Total  Symptoms  -r  ~  34,  p  <  .001. 

Reliability  of  the  TRS 

Maes  and  Anderson  report  alpha  reliability  estimates  of  .87  (1985)  and  .85  for 
the  original  32-item  version  of  the  TRS.  A  two-week  interval  test-retest  reliability 
was  estimated  at  .75.(1986).  The  estimated  alpha  reliability  reported  for  the  revised 
24-item  TRS,  as  was  used  in  this  study,  was  .84  (1986). 

Sample 

The  sample  con '.prised  167  undergraduate  students  who  were  in  the  penultimate 
year  of  their  preservice  training.  The  group  was  randomly  divided  in  half  with  the 
only  distinction  being  that  each  group  used  a  different  form  of  locus  of  control 
measure.  Two  locus  of  control  measures  were  used  because  it  was  felt  that  locus  of 
control  ought  to  be  predictive  of  teachers'  computer  use  and  while  these  two 
measures  overlapped  they  were  distinct  in  the  situations  they  addressed.  Because  of 
the  time  required  to  administer  both,  it  was  decided  to  only  atiminister  one  to  each 

half.  ,      j  •  e« 

The  undergraduates  elementary  education  majors  were  selected  s\nce  as  future 
elementary  school  teachers,  they  should  be  teaching  a  variety  of  subjects  and  not  be 
influenced  to  use  computers  by  subject  matter  which  naturally  use  computers.  For 
example,  secondary  school  teachers  typically  are  subject  matter  specialists.  Some 
subject  areas  such  as  mathematics  or  language  arts  may  emphasize  the  use  of 
computers;  the  teachers'  use  of  computers  in  teaching  may  be  largely  influenced  by 
the  prevalence  of  the  computers  use  in  the  subject  area. 

Results 

Table  3  shows  the  distribution  of  the  preservice  sample  by  expected  levels  of 
computer  use.  Preservice  undergraduate  teachers  were  instructed  to  respond  to  the 
assessment  items  according  to  their  expected  level  of  computer  use 
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Table  3 

Claaaification  of  Preaervice  Teachers 
by  Levela  of  Expected  Computer  Use 


Computer  V 

Preaervice 

% 

Levela 

Teachers 

Nonuae 

4 

2.7 

Utilization 

126 

84 

Integration 

20 

13.3 

Total 

150 

100 

Note.  17  cases  were  deleted  due  to  missing  values  or  because 
responses  to  the  LU  assessment  were  inconsistent. 


Use:  Of  the  167  subjects  participating,  17  were  not  eligible  for  classification  of  use 
according  to  the  LU  assessment— these  were  considered  missing  values.  Of  the  150 
subjects  classified  for  use,  there  were  4  who  did  not  expect  to  use  computers  at  all  for 
teaching.  The  majority -84%-- were  classified  at  the  Utilization  level,  and  13%  were 
classified  at  the  Integration  level.  The  question  raised  is  what  their  levels  of  use  will 
be  at  the  subsequent  phases  of  their  careers.  For  a  comparison  with  the  levels  of  use 
of  practicing  teachers  see  Marcinkiewicz  (1991);  it  was  found  that  the  levels  of  use 
were  inconsistent  with  the  expected  levels  of  use  of  the  preservice  teachers. 

The  descriptive  data  reported  in  Table  4  includes  the  mean,  standard  deviation, 
minimum  and  maximum  values  for  each  variable  as  well  as  the  mode,  percentage 
and  the  count  of  the  mode, 

Descriptive  Statistics 


Table  4 

Descriptive  Summary 


variable 

min 

max 

mean 

s.d. 

mode 

% 

count 
out  of 
167 

age 

20 

41 

22.7 

4.2 

20-23 

85 

142 

gender 

female 

92 

154 

experience 

0 

13 

2.9 

2.1 

2-4 

38.9 

65 

innovativeneaa 

34 

70 

53.9 

7.4 

50-55 

28.1 

47 

locua  of  control 

26 

71 

50 

10 

45-52 

26.3 

44 

perceived 

1 

7 

6.05 

1.1 

6-7 

79 

132 

relevance 

aelf- 

1 

7 

5 

1.5 

6 

35.3 

59 

competence 

ns  were 


computed  fov  all  respondents  including  those  whose  responses  to  the  LU  assessment 
were  inconsistent, 


Four  of  the  predictor  variables,  are  age,  gender,  perceived  relevance,  and  self- 
competence  are  relatively  uniform —they  have  little  variation.  This  was  anticipated 
for  this  sample  regarding  age  and  gender.  Eighty-five  percent  of  the  sample  was 
between  the  ages  of  20  to  23  and  92%  was  female.  Future  observations  will  bear  little 
information  regarding  the  influence  of  gender.  However,  any  change  in  computer  use 
for  the  group  as  a  result  of  chronological  aging  mignt  be  quite  informative. 

It  is  noteworthy  that  the  sample  rather  uniformly  tended  towards  a  perception  of 
high  relevance  of  computers  to  teaching  (perceived  relevance)  and  a  perception  of 
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high  likelihood  of  the  ability  to  use  computers  competently  for  teaching  (self- 
competence).  .  „ 

The  mean  of  years  of  computer  experience  was  2.9  (s.d.  -  2.12)  with  a  range  ol 
0  to  13.  The  mean  score  for  innovativeness  was  53.93  (s.d.  =  7.4)  with  a  range  of 
34  to  70.  The  two  teacher  locus  of  control  measures  (TRS  and  TLC)  were 
standardized  to  T-scores.  This  was  done  because  the  entire  sample  used  identical 
questionnaires  except  that  half  were  randomly  administered  questionnaires  which 
included  the  TRS  and  the  other  half  were  randomly  administered  the  TLC. 

Correlations  were  computed  for  both  halves  as  well  as  logistic  regressions  to 
determine  if  either  locus  of  control  measure  was  related  to  use.  The  correlation  of  the 
TLC  to  use  was  r  =  .  16;  p  =  .  14.  The  correlation  of  the  TRS  to  use  was  r  =  .09; 
p  =  .37.  In  a  stepwise  logistic  regression,  neither  measure  of  teacher  locus  of  control 
was  identified  as  a  predictor  for  its  reapective  half  of  the  sample.  Since  the  measures 
of  locub  of  control  were  neither  significantly  correlated  to  use  nor  were  they  identified 
as  predictors  of  use  and  because  they  had  differing  scales,  the  teacher  locus  of  control 
values  were  standardized  to  T-scores  and  the  reported  correlations  and  logistic 
regression  were  computed  for  the  entire  sample.  Hie  T-score  mean  was  50  and  the 

s  cl  Was  10. 

Intercorrelation  of  all  variables  were  computed.  (See  Table  5).  The  two  variables 
that  resulted  in  moderate  and  statistically  significant  correlations  to  use  were 
perceived  relevance  and  self-competence -r=. 28;  p  <  .01  and  r  =  23  p  <  .05 
respectively.  These  two  variables  are  complements  derived  from  expectancy  theory. 


Table  5 

Intercorrelation  a  of  All  Variables  in  the  Analysis 


use 

age 

gender 

comput. 
exp. 

innovat. 

locus  of 
control 

use 

age 

-.098 

gender 

.121 

042 

computer 

.094 

-.087 

.208 

experience 

-.119 

-.033 

innovativeness 

-.011 

.08 

locus  of  control 

.125 

.047 

-.081 

.091 

.242 
* 

perceived 

.28 

.034 

.081 

.319 

.059 

.073 

relevance 

** 

** 

aelf -competence 

.23 
* 

.138 

i06 

.264 
* 

.07 

.022 

*p  <  .05; 

**p  <  .  01; 

.  001 

perceiv'd  aelf- 
relev^nce  comp. 


.376 


Other  statistically  significant  correlations  were  years  of  computer  experience 
with  perceived  relevance  ( r  -  .319;  /;  <  .01);  and  years  of  computer  experience  with 
self-competence  (r  =  .264;  p  <  .05).  Innovativeness  was  correlated  to  the  standardized 
locua  of  control  values  which  assessed  for  internal  locus  of  control  (r  =  .242;  p  <  .05). 

The  correlation  between  perceived  relevance  and  self-competence  was  strongest 
(r  =  376;  p  <  ,001).  This  relationship  may  support  the  theorized  complementary 
nature  of  these  two  variables,  in  =  167  for  all  variables  except  use,  average  values 
were  included  tor  one  case  missing  in  each  of  age,  gender,  locus  of  control,  and 
innovativeness) 

A  logistic  regression  was  computed  to  explore  the  possible  predictiveness  of  these 
variables  to  use.  The  two  variables  identified  as  predictive  were  perceived  relevance 
and  self-competence.(/i!  =  .27; p  <  .05  and  R=  .22;;;  <  .05). 
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Future  Plana 

Because  perceived  relevance  and  self-competence  were  computed  to  be  most 
highly  correlated  with  computer  use  and  because  their  complementary  nature  is 
supported  by  expectancy  theory,  a  more  precise  investigation  of  them  and  their 
relationship  to  computer  use  will  be  undertaken  in  the  upcoming  phases  of  this 
study.  A  measure  of  self-efficacy  in  computer  use  has  been  identified.  A  measure  of 
perceived  relevance  is  being  developed. 

Teacher  locus  of  control  may  not  be  assessed  in  the  future  phases  for  two  reasons. 
First,  none  of  the  teacher  locus  of  control  scales— the  TLC  or  the  TRS— either 
individually  or  as  combined  values  standardized  in  T-scores  were  strongly  or 
significantly  correlated  to  use.  Secondly,  locus  of  control  together  with  stability  is  n 
dimension  of  the  construct  of  self-efficacy.  The  construct  underlying  the  measure  of 
self-competence  is  related  to  the  construct  of  self-efficacy.  Therefore,  it  is  anticipated 
that  assessment  of  self-efficacy  will  encompass  locus  of  control  as  well. 

We  plan  to  control  for  the  influence  of  environmental  factors.  In  the  future  phases 
of  the  study  when  the  subjects  will  actually  be  professional  teachers,  some  effects  of 
the  environment  will  be  controlled  by  limiting  the  subjects'  eligibility  for 
participation  in  the  study  by  specifying  maximum  pupil-computer  availability  ratios 
at  their  respective  schools.  The  planned  for  maximum  pupil-computer  availability 
ratio  is  44-1  following  the  precedent  of  Marcinkiewicz  (1991).  That  had  been  based 
on  national  averages  from  1987  (OTA).  However,  given  the  increased  availability  of 
computers  in  education,  the  availability  ratio  may  be  halved;  so  that,  in  order  for  a 
teacher  to  participate  in  the  study,  the  teacher  will  have  to  work  in  a  school  where 
there  is  at  least  one  computer  available  for  no  more  than  22  children.  There  is 
however,  an  aspect  of  the  environment  that  has  not  been  included  in  this  study  but 
which  has  been  proposed  in  the  theory  of  planned  behavior  (Ajzen,  1985).  That  is, 
the  element  of  subjective  nonus  which,  briefly,  means  one's  perception  of  the 
expectations  of  one's  environment.  Including  this  variable  may  be  informative  and  it 
will  complement  the  variables  currently  assessed. 

Conclusion 

An  overwhelming  percentage  of  preservice  teachers  seems  to  expect  to  use 
computers  for  teaching  if  at  least  at  the  utilization  level.  The  two  variables  of 
"perceived  relevance"  and  "self-competence"  emerged  as  being  statistically 
significantly  correlated  with  and  as  predictors  of  computer  use  .  This  result  is 
congruent  with  expectancy  theory  of  which  these  vahables  are  components  The 
results  have  encouraged  us  in  our  examination  of  these  variables  in  relation  to 
computer  use  by  teachers. 
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Introduction 

An  exploratory  study  was  conducted  examining  children's  responses  to  interactive, 
nonlinear  HyperCard  stories  and  the  children's  design  of  their  own  stories  in  HyperCard 
on  the  Macintosh  computer.  Twenty  three  fifth  graders  took  part  in  a  language  arts  training 
program  that  included  HyperCard  as  well  as  traditional  stories  in  books.  The  question  was 
posed:  How  does  HyperCard  affect  children's  understanding  of  story  structure?  What 
happens  when  the  conventions  of  story  —  beginning,  middle,  end  —  are  transformed  into 
a  nonlinear  form?  This  is  an  important  issue  since  the  research  on  reading  and  text 
comprehension  shows  that  learners  who  understand  the  conventions  of  story  structure  are 
better  able  to  understand  the  text  and  learn  to  read  and  comprehend  more  effectively  than 
their  peers  who  do  not  understand  story  structure  (Gage  and  Berliner,  1988). 
Study  Overview 

In  this  study,  a  semester-long  language  arts  training  program  utilizing  HyperCard  was 
implemented  with  a  group  of  23  fifth  graders.  This  program  consisted  of  three 
components.  First,  the  children  were  introduced  to  a  variety  of  children's  literature  read  to 
them  by  the  university  researcher.  The  researcher  explained  the  characteristics  of  story 
structure  and  the  components  of  stories,  such  as  characters  and  plot  and  discussed  the  story 
characteristics  for  the  stories  she  read  with  the  children.  Second,  the  children  were 
introduced  to  HyperCard  through  the  Amanda  Stories  (Goodenough,  1987)  which  are 
nonlinear  narratives  told  through  a  series  of  pictures  that  include  simple  animation  and 
sound  effects,  but  virtually  no  written  text.  The  computer  sessions,  with  children  working 
in  cooperative  pairs,  were  recorded  on  video.  The  children  "talked  through"  the  stories  and 
after  completing  one  of  the  Amanda  Stories,  the  children  were  asked  to  discifss  the  story 
structure  and  story  features  and  to  compare  the  story  to  book-based  stories.  In  the  third 
and  final  part  of  this  project,  the  children  wrote  and  illustrated  their  own  stories  which  were 
adapted  to  HyperCard. 

The  Amanda  Stories  are  interactive  narratives  told  through  a  series  of  pictures  thai 
include  simple  animation  and  sound  effects,  but  virtually  no  written  text.  To  move  a  story 
forward  using  HyperCard,  the  children  used  a  mouse  to  control  the  cursor  on  the  screen 
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and  to  click  on  "hot  spus"  hidden  within  each  picture.  Since  more  than  one  hot  spot  is 
hidden  within  many  of  the  pictures  appearing  on  the  screen,  there  are  a  variety  of 
possibilities  for  plot  development.  The  Amanda  Stories  are  similar  in  nature  to  wordless 
picture  books.  These  stories,  including  "Your  Faithful  Camel  Goes  to  the  North  Pole"  and 
"Inigo  at  Home"  provided  the  children  with  a  basic  understanding  of  HyperCard  and  its 
possibilities  for  storytelling  and  story  writing.  HyperCard  is  a  tool  for  designing  and 
accessing  information  flexibly  and  in  a  nonlinear  fashion  on  the  Macintosh  computer.  To 
move  a  story  forward  using  HyperCard,  the  children  used  a  mouse  to  control  the  cursor  on 
the  screen  and  to  click  on  "hot  spots"  hidden  within  each  picture,  or  card.  Since  more  than 
one  hot  spot  is  hidden  within  many  of  the  pictures  appearing  on  the  screen,  there  are  a 
variety  of  possibilities  for  plot  development. 

The  children  were  encouraged  to  write  fully-formed  stories  on  topics  inspired  by  a 
variety  of  children's  literature  read  to  them  by  the  university  researcher.  An  additional 
source  of  inspiration  for  these  fifth  graders  were  the  Amanda  Stories.  While  creating  their 
own  HyperCard  stories,  the  children  used  their  problem  solving  capabilities.  The  childen 
composed  their  stories  with  pen  and  paper  and  then  their  writing  was  transferred  to  word 
processing  files  using  the  computer.  All  of  the  children  were  given  total  access  to  the 
composing  process,  brainstorming  topics  for  writing,  sharing  their  work  in  progress,  and 
ultimately  editing  their  stories  before  publication.  The  children  created  illustrations  for  their 
stories,  using  pencil  and  paper,  and  these  illustrations  were  scanned  into  the  computer  and 
then  placed  into  HyperCard  computer  stacks.  Thus  the  children's  HyperCard  stories 
consisted  of  a  sequence  of  drawings  with  "hot  spots"  linking  them. 

At  the  end  of  the  project,  the  children  had  the  opportunity  to  view  one  another's 
HyperCard  stories  via  a  liquid  crystal  display  overhead  projector  linked  to  a  Macintosh 
computer.  Needless  to  say,  the  children  were  filled  with  a  sense  of  excitement  and 
self-satisfaction  seeing  the  results  of  their  work.  These  stories  were  compiled  in  book  form 
in  Super  HyperCard  Computer  Stories;  this  book  includes  both  the  written  stories  and 
printouts  showing  the  cards  or  pages  of  illustrations  in  the  HyperCard  stacks.  This  is  a 
unique  collection  of  narratives  written  and  illustrated  by  23  fifth  graders  during  their  very 
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first  introduction  to  HyperCard. 
Methods 

The  research  methods  used  in  this  qualitative  study  included:  1)  taking  notes  on  class 
discussion  and  student  questions  concerning  the  story  structure  of  stories  read  out  loud 
from  books;  ?.)  observation  of  student  pairs  working  through  the  Amanda  Stories  on  the 
computer  and  related  interviews  with  students;  and  3)  evaluation  of  student  stories,  both 
text  and  pictures. 

Students  worked  through  the  Amanda  Stories  in  pairs.  They  were  directed  to  take 
turns  using  the  mouse  and  "talking  through"  the  story  as  it  unfolded  on  the  computer 
screen.  The  researchers  observed  the  the  sessions  on  the  computer  and  these  sessions  were 
videotaped  for  later  analysis.  All  students  were  observed  as  they  worked  through  both 
"Your  Faithful  Camel  Goes  to  the  North  Pole"  and  "Inigo  At  Home!"  After  the  pairs  of 
students  had  completed  each  story,  they  were  asked  a  set  of  questions  concerning  story 
stricture,  including:  (1)  retell  the  story;  (2)  describe  the  main  characters;  (3)  what  was  your 
favorite  part  of  the  story?;  (4)  were  there  any  differences  between  this  story  and  the  stories 
that  were  read  out  loud  in  class?;  (5)  was  there  any  special  message?;  (6)  how  did  this 
story  make  you  feel?;  (7)  did  this  story  give  you  any  ideas  for  writing  stories  of  your  own? 
Discussion 

This  discussion  of  the  research  results  will  focus  on  the  children's  responses  to  the 
Amanda  Stories  and  their  observed  interactions  with  the  stories.  The  two  Amanda  Stories 
that  were  used  with  the  fifth  graders  provided  a  valuable  contrast.  The  first,  "Inigo  at 
Home,"  consisted  of  a  series  of  incidents  that  did  not  occur  in  any  particular  order, 
although  the  story  always  starts  at  the  same  point.  In  the  second  story,  "Your  Faithful 
Camel  Goes  to  the  North  Pole,"  the  events  clearly  had  a  fairly  linear  order  and  the  "story" 
had  an  outcome,  stated  in  the  story's  tide:  the  camel  and  its  companion  were  trying  to  get  to 
the  North  Pole.  In  most  screens  or  frames  in  this  story,  the  North  Pole  could  be  seen  in  the 
distance,  a  reminder  of  the  destination  or  goal.  The  children  responded  differently  to  the 
two  HyperCard  stories. 

As  they  went  through  "Your  Faithful  Camel  Goes  to  the  North  Pole,"  the  children 
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were  very  focused  on  getting  to  the  North  Pole.  In  some  instances,  they  were  frustrated 
when  they  were  taken  off  on  what  they  perceived  as  a  detour  from  the  main  story;  for 
example,  when  the  camel  builds  an  igloo  out  of  ice  where  the  two  travellers  spend  the 
night,  or  some  of  the  embellishing  details  that  take  place  at  the  reindeer's  house  right  before 
the  final  leg  of  the  journey.  Most  students  reported  that  their  favorite  part  of  this  story  is 
the  sleigh  ride  down  the  North  Pole  at  the  very  end  of  the  story. 

By  contrast,  with  "Inigo  at  Home,"  since  there  was  no  clear-cut  goal  or  plot,  the 
children  were  happy  to  explore  the  embellishing  details,  such  as  sounds  and  simple 
animation  sequences.  They  did  not  view  these  details  as  detours  taking  them  away  from 
the  main  line  of  die  narrative.  Instead,  the  children  perceived  the  challenge  with  this  story 
to  be  the  discovery  of  all  the  possible  details  of  action,  graphics  and  sound.  For  example, 
they  took  great  delight  with  a  spider  that  lowers  itself  down  from  the  edge  of  a  picture 
frame,  prompted  by  the  click  of  a  mouse.  With  this  story,  the  students  were  working 
through  the  stack  in  a  manner  that  was  more  in  keeping  with  the  premise  underlying 
HyperCard;  the  availability  of  powerful  nonlinear  linkages.  The  story  "reader1"  doesn't 
need  to  always  go  through  the  story  the  same  way;  instead  the  "reader,"  or  learner,  has 
control  of  a  rich  array  of  options. 

How  do  HyperCard  stories  compare  with  traditional  stories?  Bolter  (1991)  suggests 

that  hypertext  systems  such  as  HyperCard  offer  a  new  type  of  narrative  structure,  based  on 

episodes,  which  creates  a  new  kind  of  story  rhythm.  According  to  Bolter, 

The  episodes  may  be  paragraphs  of  prose  or  poetry,  they  may  include 
bit-mapped  graphics  as  well,  and  they  may  be  of  any  length.  Their 
length  will  establish  the  rhythm  of  the  story  —  how  long  the  reader 
remains  a  conventional  reader  before  he  or  she  is  called  on  to  participate 
in  the  selection  of  the  next  episode.  At  the  end  of  each  episode,  the 
author  inserts  a  set  of  links  to  other  episodes  together  with  a  procedure 
for  choosing  which  link  to  follow.  Each  link  may  require  a  different 
response  from  the  reader  or  a  different  condition  in  the  computer 
system,  (p.  122). 

Instead  of  a  structure  of  beginning,  middle  and  end  that  characterizes  traditional  stories, 
hypertext  stories  are  designed  around  the  more  flexible  structure  of  episodes.  In  the  study 
reported  here,  the  two  Amanda  Stories  that  were  featured,  "Your  Faithful  Camel  Goes  to 
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the  North  Pole"  and  "Inigo  At  Home!",  provide  a  contrast  between  these  two  structures. 
"Your  Faithful  Camel  Goes  to  the  North  Pole"  retains  a  more  traditional  story  structure 
where  there  is  clearly  a  beginning,  middle,  and  end,  whereas  "Inigo  At  Home!"  features  an 
episodic  structure.  At  the  same  time,  both  of  these  stories  are  built  upon  episodes. 
Furthermore,  both  of  these  stories  are  fairly  simple  in  structure,  largely  due  to  the  fact  that 
they  are  made  up  almost  entirely  of  pictures. 

A  contrasting  story  form  can  be  found  in  iheDiscis  stories,  including  "Cinderella"  and 
A  "Long  Hard  Day  at  the  Ranch,"  in  CD-ROM  format  designed  to  be  interfaced  with  the 
Macintosh  computer.  The  Diccus  story  series  on  CD-Rom  are  interactive  stories  that 
incorporate  hypertext  features.  These  stories  are  very  traditional  linear  narratives  but  they 
are  designed  to  offer  a  wealth  of  different  "levels"  of  information,  such  as  spoken  word 
definitions  or  a  spoken  identification  of  an  object  in  an  illustration,  when  the  mouse  is 
clicked. 

The  question  of  how  HyperCard  affects  children's  understanding  of  story  structure  i:> 
a  very  important  issue  since  the  research  on  reading  and  text  comprehension  shows  that 
learners  who  understand  the  conventions  of  story  structure  are  better  able  to  understand  the 
text  and  learn  to  read  and  comprehend  more  effectively  than  their  peers  who  do  not 
understand  story  structure  (Gage  and  Berliner,  1988).  The  research  on  reading  suggests 
that  what  the  text  presents  is  important,  but  perhaps  more  important  is  what  the  reader 
brings  to  a  learning  situation.  Without  rich  schemata  for  incorporating  the  text,  not  much 
will  be  learned.  For  example,  cognitive  studies  show  that  students  who  do  not  have  a 
schemata  for  a  story  often  cannot  remember  stories  well.  Gage  and  Berliner  (1988) 
explain: 

Some  children  enter  school  with  sophisticated  ideas  about  the  nature  of 
stories.  They  know,  for  example,  mat  stories  have  beginnings  and  ends, 
are  usually  presented  in  chronological  order,  have  a  hero  or  heroine,  put  an 
obstacle  in  his  or  her  way,  and  arrive  at  a  solution  to  a  problem.  A  child 
without  a  story  scheme  is  at  a  disadvantage.  Even  if  this  child  could  decode 
ever/  word  as  well  as  someone  who  has  a  well  developed  story  schema,  he 
or  she  would  probably  be  found  to  comprehend  less  when  tested.  Reading, 
then,  is  more  than  decoding.  There  is  no  doubt  that  teaching  decoding  and 
word  recognition  skills  —  a  practice  prevalent  in  schools  —  is  helpful  in 
making  reading  more  automatic.  But  reading  now  is  seen  also  as  a 
sense-making  process,  with  meaning  at  its  core.  (p.  307) 
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Strategies  for  remedial  reading,  based  strictly  on  having  children  read  bits  and  pieces  of 
sentences  or  do  worksheets  often  devoid  of  meaning  are  probably  emphasizing  the  wrong 
thing.  These  strategies  are  much  less  valuable  to  remedial  readers  than  is  learning  to 
understand  the  underlying  structure  (Gage  and  Berliner,  1988). 

One  implication  of  this  research  on  reading  and  text  comprehension  is  that  learners 
often  need  reinforcement  in  understanding,  and  searching  out,  the  underlying  structure  of 
any  material  they  are  presented  with,  whether  in  book  or  hypertext  format.  Without  an 
understanding  of  the  structure  of  traditional  text,  learners  are  at  a  further  disadvantage  in 
approaching  the  potentially  more  sophisticated  structure  of  hypertext.  In  this  study,  the 
researcher  explained  the  characteristics  of  stories  and  discussed  the  story  characteristics  for 
all  the  stories  presented  to  the  children,  including  both  those  read  out  loud  from  books  and 
the  Amanda  stories  in  HyperCard  on  the  computer.  Thus,  it  is  recommended  that  in 
introducing  learners  to  HyperCard  stories,  teachers  present  activities  that  help  learners  think 
about,  and  practice  using,  the  structural  features  of  stories. 

Stories  provide  a  familiar,  comprehensible  structure  for  children  encountering  the 
nonlinear  capabilities  of  hypermedia  for  the  first  time.  And  stories  provide  valuable  design 
guidelines  for  hypermedia  designers  (McLellan,  in  press).  Stories  set  up  an  expectation  at 
the  beginning,  which  is  elaborated  or  complicated  in  the  middle,  and  satisfied  at  the  end. 
"Stories  are  linked  beginning  to  end  by  the  establishment  of  an  expectation  in  the 
beginning  that  is  satisfied  in  the  end.  Thus,  good  stories  are  characterized  by  a  powerful 
principle  of  coherence.  Stories  hold  their  power  over  us  as  long  as  all  the  events  stick  to 
and  carry  forward  the  basic  rhythm  (Egan,  1986)."  Thus,  the  implication  for  hypertext 
design  is  that  the  information,  the  lesson  should  be  organized  coherently  and  meaningfully, 
with  no  extra  baggage  —  at  least  in  the  initial  presentation.  This  suggests  that  selection 
and  organization  are  more  more  important  than  comprehensiveness. 

The  combination  of  stories  and  instruction  in  a  HyperCard  environment  remains  to  be 
fully  explored.  However,  the  results  of  this  study  highlight  children's  great  enthusiasm  for 
this  kind  of  interactive  story.  These  results  also  point  out  that  a  new  computer-based  story 
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structure  is  in  the  early  stages  of  development.  Further  research  is  needed,  both  in  terms  of 
how  the  story  structure  can  be  adapted  to  take  advantage  of  the  capabilities  of  HyperCard 
and  also  in  conceptualizing  new  more  active  roles  for  learners  encountering  stories.  Alessi 
&  Trollip  (1985)  suggest  that  in  any  lesson,  it  may  be  valuable  to  encourage  the  students  to 
envision  themselves  in  a  situation  where  they  can  really  use  the  information  they  are 
learning:  stories  can  help  to  encourage  this  visualization,  especially  as  the  learners,  in  the 
role  of  explorers  or  detectives,  become  part  of  the  interactive  story.  One  type  of  explorer 
role  is  the  examination  of  story  structure  and  features.  Another  role  is  as  story  creator. 
Both  of  these  roles  were  featured  in  this  project. 
Conclusions 

The  study  reported  here  suggests  that  children  can  adapt  to  the  new  story  structure 
offered  by  HyperCard.  However,  in  introducing  children  to  HyperCard,  it  is  valuable  to 
provide  training  concerning  the  features  of  story  structure  so  that  children  have  a  firm 
understanding  of  structural  features  when  they  explore  a  new  type  of  story  structure.  Also, 
it  is  beneficial  to  have  students  practice  designing  stories  for  the  hypertext  medium,  as  was 
done  in  this  study. 

One  implication  is  that  learners  often  need  reinforcement  in  understanding,  and 
searching  out,  the  underlying  structure  of  any  material  they  are  presented  with,  whether  in 
book  or  hypertext  format.  Without  an  understanding  of  the  structure  of  traditional  text, 
learners  are  at  a  further  disadvantage  in  approaching  the  potentially  more  sophisticated 
structure  of  hypertext. 

Another  implication  is  that  it  might  be  valuable  to  design  explorations  to  help  learners 
think  about,  and  practice  using,  the  structural  features  of  stories  (Bromley,  1991; 
McLellan,  in  press).  The  hypertext  program  can  be  designed  so  that  the  learner  explicitly 
examines  story  structure,  together  with  the  structure  of  the  content  domain  and  the  way  it  is 
presented. 
Future  Research 

The  next  stage  of  this  study  will  be  to  develop  HyperCard-based  episodes  that  are 
somehow  linked,  featuring  both  pictures  and  text,  and  having  the  learners  organize  them 
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-  into  interactive  sequences  that  make  sense  to  them,  including  possible  additions  they  may 

wish  to  include.  Models  of  an  episodic  structure  include  most  of  Woody  Allen's  movies 
(Radio  Days,  Manhattan,  etc.)  and  also  the  movie  Mystery  Train  by  director  Jim  Jarmisch 
(Rosenblum  &  Karen,  1986).  In  this  proposed  research,  students  will  be  encouraged  to 
make  as  many  linkages  as  possible,  and  to  build  on  what  they  have  learned  about  story 
structure. 
References 

Alessi,  S.  M.,  &  Trollip,  S.  R.  (1985).  Computer-based  Instruction:  Methods  and 
Development.  Englewood  Cliffs,  NJ:  Prentice  Hall. 

Baecker,  R.,  &  Small,  I.  (1990).  Animation  at  the  Interface.  In  B.  Laurel  (Ed.).  The  Art 
of  Human-Computer  Interface  Design.  Reading,  MA:  Addison  Wesley. 
Bolter,  J.D.  (1991).  Writing  Space:  The  Computer.  Hypertext,  and  the  History  of 
Writing.  Hillsdale,  NJ:  Lawrence  Erlbaum  Associates. 

Bromley,  K.D.  (1991).  Webbing  with  Literature:  Creating  Storv  Maps  with  Children's 

Books.  Boston,  MA:  Allyn  and  Bacon. 

Discis  Books  (1990).  Cinderella.  [CD-ROM  disc]. 

Discis  Books  (1990).  A  Long  Hard  Dav  at  the  Ranch.  [CD-ROM  disc]. 

Egan,  K.  (1986).  Teaching  as  Storytelling:  An  Alternative  Approach  to  Teaching  and 

Curriculum  in  the  Elementary  School.  Chicago,  IL:  University  of  Chicago  Press. 

Gage,  N.L.,  and  Berliner,  D.C.  (1988).  Educational  Psychology.  Boston:  Houghton 

Mifflin  Co. 

Goodenough,  A.  (1987).  Amanda  Stories.  Santa  Monica,  CA:  Voyager  Company. 
[Computer  Software]. 

Greenberger,  M.  (Ed.).  (1990).  On  Multimedia:  Technologies  for  the  21st  Century. 
Santa  Monica,  CA:  The  Voyager  Company. 

McLellan,  H.  (In  press).  Hyper  Stories:  Some  Guidelines  for  Instructional  Designers. 
Journal  of  Research  on  Computing  in  Erinnatinn 

Rosenblum,  R.,  &  Karen,  R.  (1986).  When  the  Shooting  Stops  .  The  Cutting  Begins:  A 
Film  Editor's  Story.  New  York:  Da  Capo  Press. 

ERIC  55*  5S4 


TMei 

Instructional  Analogies  and  the  Learning  of 
Tangible  and  Intangible  Concepts 


Authors: 

Timothy  J.  Newby 
Donald  A.  Stepich 


5S5 


Instructional  Analogies  any  the  Learning  of 
Tangible  and  intangible  Concepts 

Timothy  J,  Newby 
Purdue  University 
tnwald  A.  Stepich 
University  of  Northeastern  Illinois 


An  instructional  analogy  has  tarn  defined  as  an  explicit,  nonliteral  comparison  between  No  objects,  or 
sets  of  objects,  thai  describes  their  sb  uctu'al,  functional,  ano/or  causal  similarities  (Stepich  &  Newby,  1 988b) .  An 
example  incites:  &x*tb&MC^^ 

snfltoaijjtoflugl^^  ,  .   .     .        ,  . 

Commonly  uWteed  as  instructional  tools  (e.g.,  Curtis  &  Reigeluth,  1384),  analogies  have  been  employed 
to  teach  a  variety  of  subjects,  including  science  (Cavose.  1&76:  Last,  1985;  Scheintaub,  1987),  computer 
Dfoorammino  fftumelhflrt  &  Norman,  1981),  composition  (ledger,  1977),  and  creative  problem  solving  (Gordon, 
1961) '  h  a  particularly  innovative  application,  Nichier  and  Nichier  (1986)  taught  rural  villagers  in  India  principles  of 
health  and  nutrition  by  likening  them  to  mora  familiar  principles  related  to  planting  and  tending  crops.  Their 
purpose  te  to  allow  relational  information  to  be  mapped  from  a  sou*ce  known  to  the  learner  to  one  that  is  unknown 
fvosniariou  &  Schcmmer,  1988).  ,  _   .  .  . 

Instructional  analogies  h.ive  been  slwwn  to  consist  o?  four  basic  components;  fa)  the  target  domain  (oi 
subject);  (b)  the  base  domain  (or  analog);  (c)  the  connector:  and  (d)  me  ground  (ref.).  The  target  domain  refers  to 
the  new  to-be-learned  information.  From  the  previous  example,  the  target  would  be  the  rfld.b).Qflfl.CgJl.  The  base 
domain  (truck,  from  the  example)  consists  of  information  familiar  to  the  learner  which  win  be  used  to  make  a 
comparison.  Tne  connector  is  a  verb  ohrase,  such  as  iaJihfi,  wNch  establishes  the  nature  of  the  relationsnip 
between  the  base  and  target  domains  fRumelhart  &  Norman,  1931).  Finally,  the  ground  is  a  detailed  description 
of  the  similarities,  and  possible  differences,  indicated  by  the  connector.  It  is  represented  by  the  phrase,  tim&Zl 

Theories  of  analogical  transfer  ha*e  been  developed  to  explain  how  information  from  a  base  domain  is 
used  to  facilitate  the  understanding  or  manipulation  oi  information  <n  another  unrelated  target  area  (Gentner. 
1982- 1983;  1988;  Gentner  &  Toupin,  198$,  Holyoak.  1984;  1985)  in  most  cases,  analogical  transfer  has  come  to 
be  viewed  as  a  process  of  "second  order  modelling"  (Holyoak,  1985)  in  which  a  model  of  the  base  is  used  to 
orooressively  develop  a  model  for  the  target  This  process  takes  place  through  the  mapping  of  a  limited  set  of 
properties  between  the  domains.  Central  to  this  conceptualization  is  that  prior  knowledge,  which  is  organized  and 
stored  in  the  learner's  memory,  serves  as  a  framework  or  "assimilative  context"  for  the  acquisition  of  new 
knowledge  (Glass  &  Holyoak,  1986;  Mayer,  19?9). 

Although  analogies  have  been  frequently  utilized  and  have  become  integral  parts  of  accepted  theories  ct 
instructional  design  (e.g.,  Relgeluth  &  Stein,  1983),  research  to  this  point  has  been  divided  in  terms  of  their 
effectiveness  for  the  comprehension  and  retention  of  concepts.  Drugge  and  Kass  (1978),  for  example,  found 
that  verbal  analogies  dfci  not  significantly  increase  immediate  comprehension.  Gabel  and  Snerwood  (1980)  in  a 
year-long  study  of  analogies  within  a  high  school  chemistry  curriculum  demonstrated  no  significant  improvement 
on  chemistry  achievement.  Likewise,  Bean,  Singer,  and  Cowen  (1985)  found  analogies  were  not  effective  with 
above  average  students. 

With  these  inconsistent  findings  the  limitations  of  the  analogies  and  their  parameters  for  effective  learning 
should  be  investigated  in  order  to  successfully  predict  when  their  use  would  be  beneficial  within  a  particular 
instructional  setting.  To  date,  a  number  of  factors  have  been  identified.  The  first,  and  most  prominent,  is  the 
learners'  comprehension  of  the  analogy  used  to  teach  »*s  new  content.  In  the  Gabel  and  Sherwood  (1980)  study, 
analogies  were  not  helpful  to  all  student.  However,  it  was  shown  that  as  many  as  48%  of  the  subjects  did  not 
fully  understand  the  analogies  used  to  teach  the  content.  Of  those  that  did  understand,  scores  on  the  semester 
achievement  tests  were  significantly  higher.  Similar  results  were  found  in  later  studies  designed  to  identify 
difficulties  in  chemistry  prouiem  solving  (Gabel  &  Samuel,  1986;  Gabel  &  Sherwood,  1984). 

A  second  limitation  of  analogies  is  the  tendency  to  overgeneralize.  Overgeneralization  refers  to  the 
tendency  to  include  things  In  a  category  when  they,  in  fact,  don't  belong.  Schustack  and  Anderson  (1979),  for 
example,  asked  subjects  to  read  and  recall  brief  biographies  of  fictional  characters.  When  asked  to  identify 
statements  they  had  seen  before,  subjects  showed  a  higher  frequency  of  false  recognitions  when  the  fictional 
biography  was  closely  analogous  to  the  life  of  a  famous  real  person.  In  a  recent  study  by  Halpern,  Hanson,  and 
Riefer  (1990)  subjects  receiving  analogies  which  were  designed  from  base  domains  significantly  different  than  the 
target  domains  proved  to  be  more  successful  than  those  derived  from  similar  or  near  base  domain  subject  matter. 
Their  conclusions  indicated  that  those  from  the  far  domain  required  additional  depth  of  processing  to  successfully 
complete  the  structural  mapping  and  thus  the  extra  effort  increased  their  abilities  and  subsequent  performances. 
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Another  interpretation,  however,  could  include  that  those  from  the  near  domain  may  have  had  increased  numbers 
of  mistakes  due  to  overgeneralization. 

Another  limiting  factor  appears  to  be  the  time  required  to  make  use  of  analogies.  Analogies  are  effective, 
but  may  not  always  be  efficient  as  instructional  aids.  In  two  sets  of  studies,  Simons  (1982;  1984)  noted  that 
including  an  analogy  in  printed  instructional  materials  increased  recall  and  comprehension  of  newly  learned 
information,  but  only  under  conditions  of  unlimited  study  time.  Restricting  the  amount  of  time  the  subjects  were 
given  to  read  the  materials  reduced  the  advantage  of  the  analogy  based  instruction.  According  to  Simons, 
analogy  based  materials  require  more  time  because  the  additional  information  in  the  analogy  must  be  read  and 
compared  to  the  other  information  in  the  text.  This  additional  effort  pays  off  in  subsequent  reading  of  the  same 
materials,  however.  Learners  can  often  reread  text  with  analogies  more  rapidly  than  text  without  analogies 
because  of  the  deeper  conceptual  understanding  they  gained  from  the  first  reading  (Simons,  1984). 

A  final  limiting  factor  is  the  learner's  need  for  cues  indicating  the  relationship  between  the  information  to 
be  learned  and  its  analog.  Cueing  is  particularly  important  because  learners  do  not  always  see  the  relationship 
between  an  analog  and  its  target.  As  a  result,  they  do  not  always  use  the  analog  when  performing  the  target  task 
(Gick  &  Holyoak,  1980;  1983).  Reed,  Dempster,  and  Ettinger  (1985)  tried  a  variety  of  cueing  techniques  to 
increase  the  transfer  of  information  between  algebra  word  problems.  These  included  describing  the  relevancy  of 
the  analog,  making  the  analog  solution  available  while  solving  the  target  problem  explaining  why  a  particular 
equation  was  used  to  solve  an  analog  problem,  and  matching  the  complexity  of  analog  and  target  problems.  Their 
failure  to  produce  consistent  results  demonstrates  the  difficulty  learners  have  in  applying  analogous  information, 
even  when  its  usefulness  is  highlighted. 

As  indicated  by  the  previous  studies,  the  emphasis  on  the  study  of  analogies  and  their  impact  on  learning 
has  been  focused  on  the  analogy  itself  (i.e.,  how  they  are  constructed,  cued,  and  placed  within  the  instructional 
materials).  A  second  area  of  research  however,  should  also  extend  to  the  type  of  to-be-learned  concept  or 
material  within  the  target  domain  which  is  to  be  taught.  Are  analogies  more  or  less  effective  when  the  to-be- 
learned  concepts  vary  in  difficulty,  ambiguity,  complexity,  or  abstractness?  Newby  and  Stepich  (1987)  for 
example,  have  argued  that  abstract  concepts  are  qualitatively  different  than  concrete  concepts  in  ways  that  make 
them  more  difficult  to  learn.  An  analogy,  in  their  view,  can  facilitate  learning  an  abstract  concept  by  generating  a 
"prototype  substitute"  that  can  represent  the  abstract  concept  in  memory  in  much  the  same  way  that  a  prototype 
comes  to  represent  a  concrete  concept  in  a  concept  learning  task.  Davidson  (1979)  has  also  described  analogies 
as  a  way  of  translating  abstract  information  into  a  form  that  is  more  concrete  and  imaginable  and,  therefore,  more 
easily  understood.  According  to  Simons  (1982;  1984)  this  is  the  "concretizing"  function  of  analogies. 

The  concretizing  function  was  demonstrated  in  a  lesson  designed  by  lona  (1982)  to  teach  college 
students  about  electricity.  The  more  abstract  components  of  an  electrical  system  were  likened  to  the  more 
concrete  and  imaginable  components  of  a  hydraulic  system  in  which  water  flows  from  a  hilltop  reservoir  to  a  mill  at 
the  bottom  of  the  hill.  For  example,  in  the  analogy  electrical  voltage  was  likened  to  the  distance  between  the 
reservoir  and  mill;  amperage  was  likened  to  the  rate  of  water  flow;  and  electrical  resistance  was  likened  to  narrow 
pipes  or  anything  else  that  will  obstruct  the  flow  of  water.  Other  concretizing  applications  can  be  found  in  subjects 
as  diverse  as  biology  (Cavese,  1976)  and  political  science  (Russell,  1980). 

The  present  study  was  designed  to  compare  the  effectiveness  of  instructional  analogies  given  different 
types  of  concepts  within  a  single  target  domain.  Specifically,  five  concepts  rated  as  highly  tangible  and  five  rated 
as  highly  intangible,  all  from  the  same  content  area  of  physiology,  were  selected  for  this  study.  Two  groups  of 
subjects  received  training  involving  all  concepts.  Training  for  one  group  included  instructional  analogies  for  each 
of  the  concepts;  whereas,  the  training  for  the  other  group  omitted  the  analogies.  The  main  purpose  of  the 
investigation  was  two-fold:  (a)  to  investigate  if  subjects  given  instructions  with  analogies  comprehend  and  recall 
concepts  more  effectively  than  subjects  who  receive  instructions  without  the  analogies;  and  (b)  to  study  the 
differential  effects  of  analogies  gi  jn  tangible  and  intangible  physiological  concepts. 

The  design  of  this  study  also  allowed  for  the  investigation  of  several  additional  questions  involving  the  use 
of  analogies.  For  example,  "Would  an  intervening  period  of  time  between  training  and  additional  testing  affect  the 
group  performances?";  moreover,  "Would  there  be  a  reported  change  in  the  degree  of  comprehension  and  recall 
based  on  concept  type?";  "Would  using  analogies  result  in  greater  confidence  in  learning?";  "Would  using 
analogies  result  in  greater  lesson  enjoyment?";  and  "Would  the  times  required  for  initial  learning  and/or  testing  of 
the  materials  differ  between  the  groups?". 

Method 

Subtests 

Ninety  four  subjects  (72  female,  22  male)  from  an  undergraduate  introductory  educational  psychology 
course  at  a  major  midwestern  university  were  solicited  to  participate  in  the  study.  All  subjects  volunteered  in  order 
to  meet  a  course  requirement  for  participation  in  research.  The  participants'  declared  major  fields  of  study 
included  education  (64.9%),  humanities  (14.9%)  science  (1 1 .7%),  and  physical  education  (3.2%).  Eight  subjects 
failed  to  return  for  the  two-week  follow-up  testing.  These  included  six  who  had  been  assign  id  to  the  analogy 
group  and  two  from  the  no-analogy  group. 
Instructional  materials 
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finpnept  selection.  Physiology  was  selected  as  the  content  area  for  the  study  based  on  the  premise  that 
it  is  a  concept-rich  subject  that  includes  a  range  of  concepts  from  the  very  tangible  to  the  very  intangible 
Additionally,  this  subject  matter  was  predicted  to  be  highly  unfamiliar  to  the  target  group  of  subjects.  A  physiology 
instructor  from  the  Purdue  University  School  of  Veterinary  Science  and  Medicine  and  two  of  his  graduate 
asnstants  served  as  content  experts  for  the  development  of  the  materials. 

The  content  experts  used  their  course  textbook,  physiology:  A  Renulatorv  Systems  Approach  (Strand, 
1983)  to  identify  61 1  potential  target  concepts.  This  list  was  reduced  to  32  using  the  following  rules:  (a) 
concepts  that  labeled  structures  were  eliminated  while  concepts  that  labelled  processes  were  kept;  (b)  two  word 
concepts  were  eliminated  while  one  word  concepts  were  kept;  (c)  concepts  common  in  everyday  language  (e.g. 
salivation,  respiration)  were  eliminated.  .  .  JU 

In  order  to  make  the  final  selection  of  concepts  from  this  list  of  candidates,  the  32  concepts  were  rated  by 
10  experts  (professors,  instructors,  and  advanced  graduate  students)  from  Purdue  University's  School  of 
Veterinary  Science  and  Medicine  and  the  School  of  Science.  Using  a  seven-point  semantic  differential  rating 
instrument,  each  expert  rated  each  of  the  32  concepts  (Kerlinger,  1973).  Five  concepts  rated  as  the  most 
tangible  (ossification,  parturition,  micturition,  adaptation,  and  peristalsis)  and  five  rated  as  the  most  intangible 
(disinhibition,  pinocytosis,  adsorptnn,  summation,  and  catabolism)  were  selected  for  the  study. 

Instructional  materials  development.  The  purpose  of  the  development  phase  was  to  create  the  lesson,  in 
analogy  and  no-analogy  versions,  that  would  be  used  to  teach  the  concepts  selected  in  the  preceding  stape.  In 
its  final  form,  the  lesson  included  the  following  written  materials:  (a)  an  introduction  and  instructions;  (b) 
instructional  materials  for  each  concept,  including  a  definition,  a  one-paragraph  description  of  the  physiological 
process  a  verbal  analogy  (for  the  analogy  condition  only),  a  posttest,  and  a  follow-up  questionnaire.  The 
development  and  validation  of  the  materials  followed  accepted  instructional  design  principles  (e.g.,  Dick  &  Carey, 

1985*'  The  first  step  in  creating  the  lesson  was  to  meet  with  the  content  experts  and  construct  an  analogy  for 
each  of  the  selected  concepts.  Construction  of  the  analogies  followed  the  steps  outlined  by  Stepich  and  Newby 
(1988b).  For  each  concept,  the  feature  most  important  to  comprehension  was  identified  and  one  or  more 
concrete  items  having  the  same  or  a  similar  feature  were  listed.  One  of  these  concrete  items,  likely  to  be  familiar  to 
the  learners,  was  then  chosen  as  an  analog.  The  analogy  was  completed  by  describing  the  similarities  between 
the  chosen  analog  and  the  concept.  As  an  illustrative  example,  the  feature  most  important  to  understanding  the 
process  of  peristalsis  is  the  progressive  wave  of  muscular  contraction  propelling  food  through  the  digestive  tract. 
Potential  analogs  included  extracting  toothpaste  from  a  tube  or  squeezing  ketchup  out  of  a  single-serving  packet. 
Both  analogs  were  expected  to  be  familiar  to  the  learners  and  the  ketchup  squeezing  analog  was  chosen  as  the 
more  accurate  of  the  two.  Peristalsis  was  then  described  in  terms  of  the  analog: 

"Peristalsis  is  like  squeezing  ketchup  out  of  a  single-serving  packet.  You  squeeze  the  packet 
near  one  corner  and  run  your  fingers  along  the  length  of  the  packet  toward  an  opening  at  the 
other  corner.  When  you  do  this,  you  push  the  ketchup  through  the  packet,  in  one  direction, 
ahead  of  your  fingers  until  it  comes  out  of  the  opening." 
This  process  was  repeated  for  each  of  the  other  nine  concepts.  Each  of  the  analog  or  base  domains  were 
selected  based  on  two  criteria:  (a)  their  high  degree  of  familiarity  for  the  learners  (Gabel  and  Samuel, 
1986)  and  (b)  they  were  from  a  different  or  far  domain  than  the  to-be-learned  concepts  (Halpern,  et  al., 
1990) 

Next  several  physiology  textbooks  (Holmes,  1979;  Jacob  &  Francone,  1965;  Luciano,  Vanden, 
&  Sherman,  1983;  Parker,  1984;  Schmidt  &  Thews,  1983;  Strand,  1983;  Vanden,  Sherman,  &  Luciano, 
1983)  were  used  as  information  sources  and  an  initial  draft  of  the  lesson  was  written.  This  draft  included 
an  introduction  to  the  study,  a  definition,  description,  and  analogy  for  each  concept,  and  a  follow-up 
questionnaire.  This  draft  was  then  evaluated  by  the  content  experts  and  two  experienced  instructional 
designers  to  ensure  its  accuracy,  clarity,  and  appropriateness  for  the  intended  subject  group. 
Suggestions  obtained  were  incor  orated  into  a  second  draft  of  the  materials.  The  concept  materials  were 
again  evaluated  by  the  content  e>  >erts  and  their  suggestions  were  incorporated  into  a  third  draft. 

At  this  point  the  content  experts  wrote  a  set  of  20  multiple-choice  test  items  (two  items  per 
concept).  Each  item  was  developed  to  focus  on  concept  application,  as  opposed  to  simple  recall.  All  test 
items  were  then  ordered  randomly.  The  test  directions  and  items  were  then  reviewed  by  an  instructional 
designer  and  experienced  teacher  who  had  not  previously  seen  the  materials  and  who  was  not  a  content 
expert.  This  helped  to  ensure  that  the  tests  were  clear  and  comprehensible.  Evaluative  comments  were 
incorporated  into  a  revised  version  of  the  test.  . 

The  follow-up  questionnaire  was  next  constructed.  This  consisted  of  questions  to  obtain 
bibliographic  information  (e.g.,  sex,  age,  academic  major),  a  question  asking  the  participant  to  estimate 
how  many  questions  on  the  20-item  test  they  answered  correctly,  and  a  Likert-type  rating  scale  to  indicate 
their  degree  of  enjoyment  with  the  lesson. 
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The  development  of  the  materials  was  followed  by  a  field  test  involving  24  subjects.  Each  was 
given  a  complete  set  of  materials  which  included  an  introduction  and  set  of  instructional  materials,  a 
posttest,  and  a  follow-up  questionnaire.  The  instructional  materials  differed  based  on  the  independent 
variable;  however,  tests  and  questionnaires  were  identical  for  all  subjects.  Reliability  scores,  using  the 
Kuder-Richardson  formula  (Mehrens  &  Lehman,  1984)  indicated  a  posttest  reliability  of  .68.  Following 
suggestions  given  by  the  participants,  the  wording  of  several  questions  in  the  questionnaire  was  revised 
and  the  verbal  instructions  given  to  introduce  the  study  were  incorporated  within  the  written  instructions. 
Procedures 

For  the  formal  investigation,  subjects  were  allowed  to  sign  up  for  one  of  three  scheduled  90- 
minute  periods.  Each  session  was  scheduled  in  a  university  classroom  that  could  facilitate  up  to  50 
students.  Investigators  monitored  each  experimental  session  and  were  given  a  set  of  procedures  to 
follow.  A  digital  clock  was  placed  at  the  front  of  the  room  to  provide  consistent  time  to  be  recorded.  When 
the  subjects  were  seated  the  investigator  briefly  introduced  the  study  and  distributed  the  handout 
containing  the  introduction  and  concept  lesson.  This  handout  had  been  stacked  alternating  between 
analogy  and  no-analogy  versions.  The  copies  were  then  distributed  by  rows,  effectively  randomizing 
assignment  to  the  two  experimental  conditions. 

The  concept  lesson  asked  the  subjects  to  note  the  last  four  digits  of  their  social  security  number 
(for  identification  purposes)  and  to  record  the  time  they  began  studying  the  materials.  After  they  had 
studied  the  concept  materials  for  as  long  as  they  wished,  they  were  again  asked  to  note  the  time.  The  first 
handout  was  then  returned  to  the  investigator  whereupon  a  copy  of  the  posttest  was  received.  All 
subjects  received  the  same  posttest.  Subjects  were  asked  to  record  the  4  digit  identification  code,  as  well 
as  the  starting  and  completion  times  of  the  test.  No  limits  were  given  on  the  amount  of  time  to  finish  the 
exam. 

After  the  test  was  completed,  subjects  returned  it  to  the  instructor  and  they  were  given  a  copy  of 
the  follow-up  questionnaire.  Upon  completion  and  return  of  the  questionnaire  each  subject  was 
reminded  to  return  in  14  days  at  the  same  time  and  location.  No  further  instructions  were  given  and  they 
were  free  to  leave. 

When  the  subjects  returned  in  14  days  they  were  each  given  a  second  posttest.  This  test 
consisted  of  a  short  set  of  directions  and  the  same  20  questions  (in  a  new  random  order)  of  the  initial 
posttest.  Subjects  were  also  asked  to  record  their  four-digit  social  security  code  number  and  the 
beginning  and  ending  time  of  the  test.  No  time  limit  was  given  for  the  test  and  it  was  returned  to  the 
investigator  upon  completion.  All  subjects  were  then  debriefed  and  thanked  for  their  participation. 

Results 

This  investigation  examined  four  dependent  variables:  comprehension  of  the  concepts  (both 
immediate  and  longer  term);  enjoyment  of  the  lesson;  confidence  in  learning;  and  time  required  to  study 
the  concepts  and  complete  the  posttests.  in  each  case  separate  analyses  were  completed. 

To  compare  the  comprehension  of  the  concepts  across  conditions,  the  20-question  immediate 
posttest  was  graded  for  each  subject  and  the  results  were  grouped  based  on  method  of  instruction  and 
type  of  concept.  A  mixed-factorial  analysis  of  variance  (method  of  instruction  by  type  of  concept  with 
repeated  measures)  was  performed  on  the  mean  number  of  items  answered  correctly  for  each  concept. 
As  shown  in  Figure  1,  those  subjects  receiving  instructions  with  analogies  significantly  outperformed 
those  who  did  not  receive  the  analogies  (£  (1 ,92)  « 1 0.53;  MS  *  67.92;  p.  <  .002).  No  significant 
difference  was  shown  between  the  comprehension  of  the  tangible  and  intangible  concepts;  however,  a 
significant  interaction  (£  (1,92)  ■  6.09;  MS  ■  17.28;  jj  <  .02)  between  the  two  independent  variables  was 
recorded.  Figure  1  illustrates  that  those  subjects  in  the  analogy  group  performed  at  a  slightly  higher  level 
given  tangible  concepts  than  they  did  given  intangible  concepts;  whereas,  those  in  the  no-analogy  group 
achieved  a  higher  score  given  intangible  concepts  when  compared  with  their  scores  for  the  tangible 
concepts. 
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Figure  1.  Immediate  posttest  comprehension  mean  scores  comparing  aa.  )gy  versus  no- 
analogy  trained  subjects  across  tangible  and  intangible  concepts. 
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Figure  2.  Fourteen-day  posttest  comprehension  mean  scores  comparing  analogy  versus  no- 
analogy  trained  subjects  across  tangible  and  intangible  concepts. 

The  two-week  posttest  was  examined  in  the  same  manner  as  that  of  the  initial  posttest.  As  shown  in 
Figure  2,  the  two  groups  attained  the  same  mean  scores  as  reported  for  the  initial  posttest  for  the  tangible 
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concepts  but  decreased  in  their  comprehension  for  the  intangible  concepts.  A  significant  difference  was 
again  recorded  based  on  the  method  of  instruction  as  those  receiving  analogies  significantly 
outperformed  those  not  receiving  such  instructions  (£  (1 ,84)  -  9.43;  MS  «  67.37;  c  <  .003).  In  this  case 
however,  there  was  a  significant  difference  between  the  tangible  and  intangible  concepts  (£  (1 ,84)  « 
7.17;  MS  -  22.349;  q  <  .009),  but  no  significant  interaction  was  reported. 

Enjoyment  of  the  lesson  was  measured  by  a  single  questionnaire  item  in  which  the  subjects  were 
asked  to  rate  how  much  they  had  enjoyed  the  lesson  on  a  scale  from  1  (not  at  all)  to  5  (a  lot).  A  significant 
difference  was  found  between  the  analogy  (M  ■  3.02)  and  the  no  analogy  (M  -  2.59)  conditions  using  a 
two-tailed  t-test  (t  (92)  *  1.869;  c  <05). 

Confidence  in  learning  was  measured  by  asking  the  subjects  to  predict  how  many  items  they  felt 
they  had  answered  correctly  on  the  initial  posttest.  Means  for  the  analogy  and  no-analogy  conditions  were 
14.19  and  13.55  respectively.  A  two-tailed  t-test  indicated  no  significant  difference  between  the 
analogy/no-analogy  predictions. 

In  order  to  measure  the  final  variable,  time,  all  subjects  were  asked  to  record  the  time  they  started 
and  finished  both  the  lesson  and  the  two  posttests.  The  time  spent,  in  minutes,  was  then  tallied  for  the 
analogy  and  no-analogy  conditions.  Two-tailed  t-tests  were  performed  on  the  mean  study  times  and  the 
mean  test  times.  A  significant  difference  was  found  in  the  mean  study  times  (1  (92)  «  1 .953;  p.  <  .05)  but 
not  for  either  of  the  testing  periods.  The  mean  study  times  for  the  analogy  and  no-analogy  conditions 
were  12.04  minutes  and  10.55  minutes,  respectively. 

Discussion 

From  the  measurement  of  the  four  dependent  variables,  comprehension,  enjoyment, 
confidence,  and  time,  several  important  findings  should  now  be  discussed.  First,  the  results  indicate  that 
analogies  had  a  beneficial  effect  on  the  comprehension  of  unfamiliar  concepts.  In  other  words,  subjects 
who  received  instructions  which  included  analogies  scored  significantly  higher  on  the  immediate  posttest 
than  those  who  did  not  receive  training  which  included  the  analogies.  Moreover,  this  difference  in 
comprehension  was  sustained  during  the  two-week  posttest.  Even  though  the  learners  were  not 
prompted  to  recall  or  use  the  analogies  in  any  way,  comprehension  scores  indicate  a  difference  between 
the  two  groups  remained  even  after  the  14-day  interval.  The  use  of  analogies  appears  to  be  an  effective 
instructional  strategy  which  increases  the  immediate  and  long  term  comprehension  of  concepts. 

Conclusions  about  the  influence  of  analogies  on  the  comprehension  of  different  types  of 
concepts  in  the  present  experiment  are  strengthened  by  the  fact  that  the  subjects  in  the  analogy 
condition  were  neither  trained  in  the  use  of  analogies  nor  cued  to  use  the  analogies  they  were  given.  Gick 
and  Holyoak  (1980;  1983)  and  others  (e.g.,  Reed,  Dempster,  &  Ettinger,  1985;  Schustack  &  Anderson, 
1979)  have  shown  that  training  and  cueing  are  essential  aspects  of  using  analogies  effectively  in 
instruction.  The  subjects  in  the  present  experiment,  however,  were  not  given  practice  in  using  analogies 
or  cues  to  use  analogies  in  recalling  the  concepts  for  the  test.  In  spite  of  this,  subjects  in  the  analogy 
condition  outperformed  subjects  in  the  no-analogy  condition  on  both  posttests.  This  indicates  that 
training  and  cueing  may  not  be  necessary  and  that  analogies  facilitate  concept  learning  even  when  the 
subjects  are  neither  trained  nor  cued.  Additional  research  is  needed  to  further  explore  the  contribution 
training  and  cueing  make  to  the  effectiveness  of  analogical  instruction  as  well  as  the  best  methods  for 
providing  such  training  and  cueing. 

A  second  relevant  finding  from  this  investigation  was  the  degree  of  effectiveness  of  analogies 
based  on  the  concepts  being  either  tangible  or  intangible  in  nature.  Although  during  the  immediate 
posttest  the  analogy  group  significantly  outperformed  the  no-analogy  group  given  either  type  of  concept, 
the  analogy  group  showed  no  significant  difference  in  the  comprehension  of  the  tangible  versus  that  of 
the  intangible  concept  types.  This  was  not  the  case,  however,  after  the  14-day  interval  had  elapsed.  In 
that  instance,  while  the  analogy  group  still  significantly  outperformed  the  no-analogy  group  for  both  types 
of  concepts,  the  comprehension  of  the  intangible-type  concepts  decreased  significantly  for  both  groups. 
Although  the  analogy  instruction  remained  more  effective,  the  concurrent  decrease  indicates  the 
possibility  of  interference  being  greater  for  those  concepts  of  a  less  tangible  nature.  Several  authors  have 
pointed  out  the  differences  between  concept  types  and  the  increased  difficulty  of  those  of  an  intangible 
nature,  however,  further  research  focusing  on  the  potential  interference  differences  is  needed  (Newby  & 
Stepich,  1987;  Reed  &  Dick,  1968). 

A  final  point  that  should  be  considered  in  the  discussion  of  the  concept-type  differences  is  that  of 
the  interaction  effect  shown  to  occur  during  the  initial  posttest.  As  indicated  in  Figure  1 ,  the  no-analogy 
group  initially  scored  higher  for  intangible  concepts  than  they  reported  for  the  tangible  concepts  while  the 
opposite  was  true  for  the  analogy  group.  Such  results  by  the  no-analogy  group  contradict  previous 
investigations  as  well  as  their  own  14-day  interval  posttest  results,  and  thus  may  be  an  indication  of 
spurious  results  (Royer  &  Cable,  1976;  Reed  &  Dick,  1968).  However,  th  s  result  may  also  be  an 
indication  of  the  overall  effectiveness  of  the  training.  For  each  individual  concept  the  learner  received  a 
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definition  and  a  description.  It  may  be  that  those  different  training  lessons  for  the  intangible  concepts 
were  initially  clearer  or  more  readily  understood.  These  conclusions  are  questionable  however,  because 
of  similar  effects  not  being  revealed  within  the  data  from  the  analogy  group  and  because  of  no  interaction 
seen  aftger  the  two-week  interval. 

A  second  factor  measured  during  this  investigation  was  enjoyment  for  the  lesson  as  perceived  by 
the  learners.  Following  training  and  the  initial  posttest,  all  subjects  were  asked  to  rate  the  degree  to  which 
they  enjoyed  the  lesson.  The  results  indicate  the  use  of  analogies  positively  influenced  the  subjects' 
rated  enjoyment  of  the  lesson.  That  is,  analogies  may  be  valuable  in  instruction  because  they  increase 
the  learnrs'  enjoyment  of  learning.  Keller  and  Kopp  (1987)  and  Newby  (in  press)  have  pointed  out  that 
familiarity  helps  learners  see  the  relevance  of  what  they  are  learning  which  increases  their  motivation  to 
learn.  Analogies  connect  new  concepts  to  familiar  objects  and  events  and,  thus,  may  increase  the 
learners'  motivation  by  increasing  the  relevance  of  the  instruction.  The  strength  of  this  conclusion  is 
limited,  however,  because  it  is  based  on  a  single  subjective  measure.  Additional  research  is  needed  to 
explore  the  effects  of  using  analogies  on  more  objective  measures  of  enjoyment. 

The  third  variable  measured  the  perceived  confidence  subjects  had  in  their  comprehension  of  the 
concepts.  The  results  show  that  using  analogies  did  not  affect  the  subjects'  confidence  in  their  learning. 
This  is  contrary  to  what  might  be  expected,  especially  in  light  of  the  observed  differences  in 
comprehension  between  subjects  in  the  analogy  and  no-analogy  conditions.  One  possible  explanation 
for  this  result  is  that  the  subjects  perceived  the  concepts  as  so  difficult  that  they  saw  little  chance  of 
success,  even  with  analogies  to  help  them,  and  so  predicted  a  lower  score  than  they  actually  obtained. 
Another  possibility  is  that  confidence  is  not  closely  related  to  actual  performance.  The  relationship 
between  analogy  and  confidence  is  an  important  issue  because  confidence  is  an  important  aspect  of 
motivation  (Keller,  1983).  Additional  research  is  needed  to  explore  the  relationship  and  to  determine  how 
analogies  might  be  used  to  increase  the  confidence  of  the  learners. 

The  final  measured  variable  included  the  amount  of  time  they  used  to  both  study  the  concepts 
and  complete  the  posttest.  The  results  show  that  subjects  in  the  analogy  condition  used  significantly 
more  time  studying  than  subjects  in  the  no-analogy  condition,  but  did  not  require  significantly  more  time  to 
complete  the  posttest.  These  results  are  consistent  with  the  findings  of  Simons  (1982;  1984)  in  which 
analogies  may  be  more  time-consuming,  as  well  as  more  effective,  than  other  instructional  strategies.  It  is 
the  efficiency  of  the  instruction  rather  than  its  effectiveness  that  is  at  issue  here.  Even  when  they  are 
equal  in  length,  lessons  with  analogies  require  more  time  than  lessons  without  analogies  because  the 
analogies  require  the  learners  to  process  the  connection  between  the  target  and  analog  in  addition  to 
reading  the  content.  Simons  notes  that  there  is  a  trade-off  between  the  increased  time  needed  to  use 
analogies  and  their  benefits  in  performance.  The  results  of  the  present  experiment  support  Simons' 
conclusion.  However,  the  strength  of  this  support  is  limited  by  the  fact  that  the  lessons  used  in  the 
present  experiment  were  unequal  in  length.  For  the  analogy  condition  the  analogies  were  simply  added 
to  the  descriptive  paragraphs  for  each  concept,  which  means  that  the  subjects  in  the  analogy  condition 
had  more  content  to  read.  Subjects  in  the  analogy  condition  took  more  time  to  study  the  concepts,  but 
their  comprehension  was  improved.  The  exact  location  of  the  balance  point  is  unclear,  however,  and  may 
be  a  purely  subjective  decision  to  be  made  by  the  individual  teacher  or  learner.  Additional  research  is 
needed  to  further  investigate  the  relationship  between  study  time  and  comprehension  in  concept 
learning  through  analogical  instruction. 

In  conclusion,  several  lines  of  reasoning  converge  to  suggest  that  analogies  are  potentially 
powerful  instructional  tools:  (a)  their  pervasiveness  in  both  everyday  and  instructional  communication;  (b) 
anecdotal  evidence  of  their  influence  in  scientific  discoveries  throughout  history;  and  (c)  empirical 
evidence  of  their  effectiveness  in  a  variety  of  learning  tasks.  However,  what  has  been  written  about 
analogies  is  almost  entirely  descriptive  in  nature.  There  is  little  prescriptive  information  and,  as  a  result,  few 
guidelines  for  using  analogies  in  instructional  practice.  Newby  and  Stepich  (1987)  have  taken  a  step 
toward  filling  this  gap  by  suggesting  a  set  of  procedural  guidelines  for  both  creating  and  utilizing  analogies 
within  instruction.  The  present  investigation  was  undertaken  as  an  empirical  test  of  one  of  their 
prescriptions:  that  analogies  are  effective  given  intangible,  as  well  as  tangible  concepts.  The  primary 
finding  confirmed  this  assumption.  Secondarily,  the  study  found  that  using  analogies  affects  the 
enjoyment  of  the  learners  but  not  their  perceived  confidence. 
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INTRODUCTION 

I  have  corns  to  the  conclusion  (Nichols,  1990,  1991)  that  several  aspects 
of  educational  technology,  especially  educational  biotechnology  (EBT),  are 
harmful  to  people  and  to  the  environment.  Something  inside  tells  me  that 
educational  technologies  will  eventually  lead  to  harms  that  far  out-weigh  any 
purported  advantages  of  the  technology.  And  I  conclude  that  this  Is  morally 
unacceptable.  One  main  issue,  then,  is  presenting  publicly  this  danger  and  this 
morality  so  thotpthers  might  consider  and  resist  educational  technology  that  does 
harm.  This  paper  is  just  such  a  presentation.  And  by  "others,"  I  mean  that  I 
especially  want  to  encourage  learners  and  teachers  to  think  and  act  more  wisely 
if  they  conclude  that  the  prospect  of  e  future  with  EBT  Is  somehow  wrong, 

In  order  to  begin  to  get  at  a  discussion  of  educational  biotechnology  and 
morality,  the  first  sections  of  the  paper  describe:  what  could  be  Immoral  about 
educational  technology  (including  EBT),  assumptions  guiding  my  current  search 
for  answers,  characterizations  of  EOT  and  related  ideas,  and  professional 
literature  about  the  issues  raised  here. 

!n  K  je  port  though.  I'll  examine  the  critical  theory  of  Habermas, 
Rortu/s  liberal  ironist,  end  Barrett's  moral  will.  Though  Habermas  Is  the  one  of 
the  three  to  be  most  closely  associated  with  critical  theory,  his  Ideas  of 
communication,  language,,  morality,  end  rationality  lead  me  almost  Inevitably  not 
only  to  some  light  put  to  Rorty.  Rorty's  theory  of  communication  and  morality 
strongly  contradict  Habermas'  notion  of  "the  Ideal  speech  situation,"  where 
perfect  communication  and  morality  are  possible.  And  when  both  men,  as 
brilliant  ts  they  are,  leave  me  short  of  being  able  to  develop  my  own  moral 
principles  In  conjunction  with  educators,  a  turn  to  the  existentialist  spirit  of 
Barrett  Is,  in  essence,  inevitable.  His  ideas  about  human  faith  and  will  and  their 
associations  with  morals  and  technology  are  particularly  Illuminating.  We  will 
turn  to  these  thinkers  before  much  longer. 

WHAT  COULD  BE  IMMORAL  ABOUT  EDUCATIONAL  TECHNOLOGY? 

Some  readers  are  asking  "How  could  It  be  that  educational  technology 
might  be  harmful  and  immoral?"  This  Idea  is  nearly  Incomprehensible  to  many 
of  us.  After  all,  this  prospect  of  the  "negative"  aspects  of  technology  goes 
against  the  proclamations  of  Apple  Computer  Corporation's  John  Sculley,  against 
many  local  school  boards,  and  against  your  Sunday  paper's  glowing  report  on 
"The  Latest  Gizmo-Tech  Revolution  for  Your  Child's  Education  And  A  Swell 
Future."  I  want  to  Indicate  just  three  basic  lines  along  which  educational 
technology  just  might  be  shown  to  be  injurious. 

First,  technology  may  Impede  learning  by  constricting  options  for 
learners  and  parents.  The  growing  accountability  movement  in  American 
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schooling  uses  technologies  (Including  standardized  testing  and 
bureaucratization)  more  and  more  to  Impose  widely  prescribed  curricula  and 
methods  on  Individual  schools  or  districts  (T aylor,  1986).  To  believe  we  can 
make  good  learners  with  a  wealth  of  stored  knowledge  is  to  say  that  learners  can 
not  be  responsible  for  their  own  selections  of  appropriate  content  and  learning 
methods— and  so  very  few  are  responsible,  at  least  according  to  the  national  tests 
that  show  us  we  are  falling. 

Hardlson  (1989)  even  goes  so  far  as  to  speculate  that  technology  is  the 
new  and  predominant  entity  on  earth  and  In  the  heavens,  In  Hne  to  very  quickly 
put  humans  In  a  position  where  we  will  communicate  less  and  even  drop  into  a 
kind  of  universal  stupor  which  Is  somehow  pleasant  to  us.  Of  course  Hardlson 
implies  that  these  possibilities  are  mostly  good.  They  are  Interesting,  natural, 
and  milestones  of  wonderful  progress.  I  believe  I  disagree. 

Soclo-culturally,  there  Is  quality  research  which  indicates  that 
oppression  of  less  advantaged  people  Is  exacerbated  by  computers  In  education. 
Sutton  (1991)  shows  the  ways  in  which  uses  in  American  schools  indicate  that 
computer  use  during  the  1980s  did  not  bring  education  closer  to  equal 
educational  opportunity.  Rather,  It  maintained  and  exaggerated  existing 
Inequalities  In  education  input,  processes  of  computer  learning,  and  output. 
Poor,  female  and  minority  students  had  less  access  to  computers  at  home  and,  in 
addition,  less  access  to  computers  at  schooL.Poor  and  minority  students  were 
more  likely  to  use  computers  for  drill  and  practice  than  were  middle-class  and 
White  students,  and  females  outnumbered  males  in  word  processing  but  were 
underrepresented  In  programming.  Teachers,  while  concerned  about  equity,  held 
attitudes  which  hindered  access:  They  believed  that  better  behaved  students 
deserved  computer  time  and  that  the  primary  benefit  of  computers  f  jr  low- 
achieving  students  was  mastery  of  basic  skills....Thus,  children  who  were 
minority,  poor,  female,  or  low  achieving  were  likely  to  be  further  behind  after 
the  Introduction  of  computers  In  school s....These  inequalities  were  found  In  the 
U.S.A.,  Great  Britain,  Australia,  Canada,  and  New  Zealand  (pp.  494) 

Thirdly,  technology  and  educational  technology  are  ecologically 
destructive  first  cousins.  I  hope  I  do  not  need  to  remind  anyone  of  the  Incredible 
destruction  associated  with  technology  In  general,  but  in  the  film  After  The 
Warming  (Sattln,  1989),  James  Burke  predicts  many  decades  of  social  and 
ecological  upheaval  and  death,  even  If  we  could  somehow  Immediately  reverse  the 
damage  we  are  doing  presently.  Which  we  most  likely  can  not.  Education  teaches 
about  the  wonderful  possibilities  of  technology  and  about  the  desirability  of  using 
technology  to  control  all  aspects  of  our  lives,  from  chl.d  birth  to  dying,  and  so 
encourages  destruction.  How  many  millions  of  personal  computers  hav6  been 
purchased  in  the  last  decade  alone?  Do  they  use  the  same  amount  of  electricity  as 
a  town  of  over  300,000  or  1,000,000  people?  They  use  a  lot  of  energy,  most  of 
it  being  fueled  by  carbon-based  materials  that  pollute.  And  just  where  does  all 
the  used  plastic  In  computers  go  when  Its  discarded? 

So  these  effects  are  bad,  but  what  could  make  them  immoral  too,  how  could 
we  be  evil  In  this  state  of  affairs?  I  suggest  that  because  we  now  know  the 
extent  of  negative  possibilities,  we  are  intentionally  choosing  to  continue  on  the 
path  of  destruction  which  is  needless  and  wanton.  Intentlonality  Is  a  critical 
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point  on  which  to  make  many  decisions  about  morals.  It  is,  for  instance,  one  of 
the  principle  bases  of  legal  systems;  if  you  know  ahead  of  time  of  the 
consequences  of  an  act  and  knowingly  procede  with  the  act,  you  are  all  the  more 
guilty.  The  path  of  technology  Is  littered  with  Immorality. 

Additionally,  and  reflecting  a  view  that  Is  central  to  this  paper,  those  few 
people  In  positions  to  make  choices  that  affect  the  world's  population  do  not 
subscribe  to  the  moral  principle,  In  cultural  morality  at  least,  that  everyone 
should  have  a  free,  Informed,  reasonable,  and  uncoerced  say  In  what  Is  and  is  not 
moral  about  technology.  Vltually  none  of  us  hod  a  direct,  meaningful  say  In  the 
exorbitant  Increase  In  America's  nuclear  arsenal?  Do  you  have  a  direct  say  in 
how  to  clean  up  the  devastation  caused  by  this  technology?  Or  will  you  just  wait 
passively  to  see  what  happens?  We  do  not  have  very  much  of  a  democracy  In  this 
sense.  To  the  extent  that  almost  none  of  us  participate  meaningfully  In 
widespread  discussions  about  these  Issues,  and  powerful  people  art  not  looking 
for  ways  to  increase  our  power,  the  few  powerful  people  could  be  construed  as 
Immoral. 

I  canonly  speculate  that  engineering  of  our  minds  and  bodies  will  only 
worsen  the  kinds  of  techno-moral  dilemmas  In  which  we  now  find  ourselves. 
ASSUMPTIONS 

Several  assumptions  guide  this  search  to  a  more  moral  technology  use. 
First,  I  assume  that  educational  biotechnology  (EBT)  is  dangerous  and  that  many 
people  with  the  desire  and  the  ability  to  do  so  will  use  drugs,  neural  surgery, 
psychological  training,  and  genetic  manipulation  to  Improve  memories,  thinking 
skills,  and  learning  attitudes  of  their  children. 

It  Is  also  an  assumption  that  critical  theory  and  literature  related  to  It 
will  help  to  show  the  way  to  a  moral  condition  In  this  natter.  The  ideas  of  the 
three  theorists  discussed  here  address  morals  an<1  over-extended  rationalization 
In  a  lively  and  current,  If  not  widespread,  fashion  that  Is  hard  to  find  anywhere 
else. 

Of  course  I  assume  (hope?)  that  we  will  not  destroy  ourselves  with  war 
weaponry  orecological  suicide  before  we  are  faced  with  the  problems  to  which  I 
allude  and  before  we  have  a  chance  to  reach  some  more  or  less  common  moral 
agreement  about  a  widespread  human  stance  in  relation  to  these  Issues.  To  make 
the  points  herein  is  to  have  at  least  a  modicum  of  hope  in  the  face  of  a  future  that 
is  quite  likely  to  be  very  harsh  for  us  and  the  earth. 

I  believe,  too,  that  current  educational  technologies  won't  positively 
revolutionize  learning  In  major  ways.  History  supports  this  assumption 
(Saettler,  1990).  Further,  only  rarely  do  the  most  widely  disseminated  calls 
for  educational  reform  and  the  reforms  themselves  seem  likely  to  use  the  fullest 
capabilities  of  technology;  saturated  use  of  technology  is  rarely,  if  ever, 
recommended  by  national  commissions  and  the  like. 

I  assume,  also,  that  In  the  next  several  years  we  still  will  be  failing 
miserably  to  educate  ourselves.  I  do  not  mean  necessarily  that  the  failure  will  be 
In  terms  of  current  conceptions  of  success,  which  focus  on  mental  facts, 
concepts,  and  generalizations  or  on  the  societal  needs  of  schooling  for 
nationalistic  economic  dominance.  I  mean  that  our  formal  education  structures 
will  not  help  us  to  avoid  a  crumbling  ecology,  a  starving  world  population,  losses 
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of  political  rights,  and  Injuries  of  selfhood. 

Perhaps  most  Importantly,  I  assume  that  any  danger  of  EBT  Is  part  and 
parcel  to  the  basic  problem  of  a  too  rational  way  of  life  (See  Sarason,  1 99 1 
about  education  and  rationalization).  This  Is  to  say  we  believe  too  strongly  that 
our  logical-Instrumental  selves  can  overcome  all  the  problems  with  which  we 
think  we  are  faced,  from  poverty  to  pimples,  even  though  the  evidence  Is  that  a 
dominant  rationality  is  too  often  as  destructive  of  the  human  spirit  and  existence 
as  it  Is  healthy.  Not  only  Is  this  conscious  self  too  mentally  and  emotionally 
subjugating,  but  It  Is  also  physically  detrimental.  As  Ellul  (1990)  concludes  in 
The  Technological  Bluff,  the  power  of  technology  means  a  "subordination  of  the 
Individual  to  machines  [and  other  technologies!  over  which  there  Is  less  and  less 
control"  (p.  1  58). 
CHARACTERIZATIONS 

The  meaning  of  educational  biotechnology  (EBT)  is  not  pat,  but  It  has  been 
described  as  "the  study  and  application  of  scientific  and  other  organized 
knowledge,  processes*,  and  products  to  the  physical  state  of  humans  for  the 
purpose  of  creating  changes  in  learning"  (Nichols,  1991,  p.  128).  For 
example,  when,  in  utero,  a  child's  genes  are  engineered  to  produce  a  person  with 
virtually  no  likelihood  of  being  "hyperactive"  In  school,  this  Is  EBT. 

Let  me  briefly  characterize  "morality"  and  distinguish  it  to  some  basic 
degree  from  "ethics."  A  basic  definition  of  morality  Is  to  call  it  a  condition 
wherein  we  make  judgments  "of  goodness  and  badness  of  human  action....'Moral' 
pertains  to  personal  behav1or...measured  by  prevailing  standards  of  rectitude. 
'Ethical'  approaches  behavior  from  a  philosophical  standpoint;  It  stresses  more 
objectively  defined,  but  essentially  Idealistic,  standards  of  right  and  wrong,  such 
as  those  applicable  to  the  practices  of  lawyers,  doctors,  and  businessmen" 
(American  Heritage  Dictionary,  1979,  p.  852). 

Ethics  Implies  the  codes  or  laws  that  derive  from  moral  principles  and 
that  are  more  or  less  situation  specific.  Since,  In  my  judgment,  practically  no 
one  is  considering  the  Issues  I  raise  here,  and  since  no  widespread  conversation 
(conversation  that  closely  reflects  the  activities  and  essences  of  the  three 
authors  examined  here)  about  EBT  and  morals  is  occurring,  I  would  be  arrogant 
and  misguided  (Indeed,  Immoral  and  unethical)  at  this  point  to  attempt  anything 
but  leaving  the  details  of  an  ethical  system  for  future  conversations.  Besides, 
I'm  cautious  about  any  kind  of  system,  as  you  may  discern  shortly.  Also,  the 
conversations  must  start  among  many  people,  not  just  among  formal 
philosophers,  educators,  and  technologists.  So  I  like  the  Idea  that  morals  tend  to 
apply  more  to  the  goodness  and  badness  of  personal  behavior,  while,  at  the  same 
time,  the  notion  of  morals  alms  for  a  greater  public  good.  Further,  If  the 
conversations  don't  happen  of  our  accord,  we  unavoidably  and  more  harshly  will 
be  faced  with  these  Issues.  And  if  the  talk  doesn't  happen  at  the  common  level  it 
will  not  take  hold  generally. 

Consequently,  this  paper  contains  less  about  specific  ethical  codjs  than  It 
does  about  what  the  conversation  on  morality  might  look  like,  what  Issues  In  the 
conversation  could  be  most  meaningful,  who  ought  to  participate,  what  moral 
principles  might  emerge,  and  how  educational  technologies  fit  into  larger  views 
of  technology  and  morals.  In  other  words,  you  will  not  find  exact  ethics 
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directions  for  living,  but  some  speculations  about  getting  started  with  the 
pressing  conversations  about  the  morality  of  EBT. 

Turning  to  the  notion  of  "rational,"  it  is  meant  here  to  be  the  broadest 
concept  of  human  logical-instrumental  thinking  and  doing.  I  am  referring  to 
nearly  any  mode  of  human  existence  whereby  we  use  logical  elements  of  our 
thinking  to  change  something  in  our  world.  Systematically  applying  a  formal 
instructional  design  and  development  model  to  the  task  of  educating  people  Is  a 
rational  activity.  "Rationalistic"  Is  the  extended,  most  intense  form  of  the 
rational.  Notions  such  cs  "empiricism"  and  "positivism"  are  meant  as  very 
closely  related  sub-concepts  of  "logical-Instrumental,"  Inasmuch  as  they  are 
more  developed,  specific  manifestations  of  rationality. 

About  critical  theory,  though  I  suggest  It  comes  up  short  In  some  ways,  I 
find  It  to  be  a  useful  foundation  to  my  search  for  morals  associated  with 
educational  biotechnology.  Many  critical  theorists  assume  that  "each  historical 
situation  Is  a  distortion  of  the  Utopian  vision  that  was  the  normative  basis  [a 
moral  basis]  for  the  existing  social  structures  and  beliefs"  (Ewert,  1 991,  p. 
355).  These  theorists  believe  that  individuals  and  societies  can  be  better,  and 
that  being  better  Involves  exposing  the  meanlnglessness  of  an  over-extended 
rationality,  whether  this  meanlnglessness  Is  found  In  science,  economics, 
politics,  mass  media,  morals,  or  education.  Rationalization  In  all  areas  of  life 
has  led  to  erosions  of  community,  self,  language,  biosphere,  and  hope,  as 
Habermas  will  help  to  show  In  a  later  section  of  this  paper. 

Thomas  McCarthy  (1991)  describes  aspects  of  critical  theory  as: 
—Challenging  the  notion  of  pure  reason;  showing  Its  changeability  depending  on 
culture,  history,  and  power  in  which  It  is  embedded. 

—Likewise,  the  theory  rejects  the  "Cartesian  picture  of  an  autonomous  rational 
subject"  who  tries  to  control  the  world. 

—  In  critical  theory  the  practical  takes  precedence  over  the  theoretical,  but 
theory  Is  bound  to  the  practical. 

—Knowledge  is  not  disembodied  from  the  rest  of  existence;  still,  though,  a 
distanced  or  objecti voting  understanding  of  knowledge  Is  needed. 
—Established  human  sciences,  scientifically  trained  experts,  and  rationalization 
all  come  In  for  close  analysis  by  critical  theorists. 

—A  major  purpose  of  critical  theory  Is  to  make  problematic  what  is  t^ken  for 
granted  In  culture  (power  relationships,  often)  so  that  a  degree  of  social  justice 
can  be  had.  (p.  43) 

And  the  critical  theorists  of  the  early  Frankfurt  School  "believed  that  the 
appropriate  method  of  critique  was  immanent  critique,  which  proceeds  through 
forcing  existing  views  to  their  systematic  conclusions,  bringing  them  face  to  face 
with  their  incompleteness  and  contradictions,  and,  ultimately,  with  the  social 
conditions  of  existence"  (Young,  1990,  p.  18).  This  belief  Is  a  recurrent  theme 
in  critical  theory  even  today. 

Without  Inferring  too  much,  then,  we  could  say  critical  theory  leads  to 
the  conclusion  that  rational-technological  aspects  of  humans  lead  to  human 
alienation  and  to  ecological  peril,  as  Habermas  will  help  to  explain. 
BACKGROUND  LITERATURE 

Many  people  have  spoken  about  technology  in  general  and  its 
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moral/ethical  implications.  Ellul  ( 1 990),  for  instance,  speaks  to  these  issues. 
He  talks  of  how  technology  is  more  and  more  surrounding  us  and  says  "This 
encirclement  or  outflanking  of  people  and  society  rests  on  profound  bases  (e.g.,  a 
change  in  rationality)  and  the  suppression  of  moral  judgment,  with  the  creation 
of  a  new  ideology  of  science"  (p.  19).  In  relation  specifically  to  learning,  Ellul 
( 1 990)  says  that,  "Pedagogy  (learning  how  to  learn)  Is  a  central  feature  of  this 
technislzlng  of  instruction,  which  Implies  a  culture  of  practical  Intelligence  In 
the  place  of  reflective  or  critical  intelligence"  (p.  136). 

And  many  people,  Strike  and  Soltls  ( 1 985)  for  instance,  have  addressed 
the  issue  of  morals  and  ethics  In  education.  The  Moral  Dimensions  of  Teaching 
(Goodlad,  Soder,  &  Sirotnik,  1990)  Is  a  good  example  of  the  ongoing  concern  for 
ethics  and  morals  within  the  professional  education  community.  In  Dimensions, 
Soder  takes  a  brief  look  at  the  "new  technology  of  schooling"  (p.  64),  but  he 
Implies  that  new  technologies  like  EBT  are  not  likely  to  proliferate,  and  besides, 
it's  not  clear  to  me  exactly  what  his  chapter  has  to  do  with  technology  and  morals. 
It's  mostly  about  the  professionalization  of  teachers. 

In  terms  of  volume,  much  has  been  written  on  critical  theory  and 
education.  Gerry  D.  Ewert  (1991)  and  Robert  E.  Young  (1990)  are  authors  of 
instances  of  this  genre.  Further,  there  is  a  sense  in  which  critical  theorists 
generally  examine  technology  very  closely,  when  the  topic  Is  rationalization  and 
the  science,  bureaucracy,  social  hegemony,  and  pollution  that  accompany  it.  Carr 
and  Kemmis  (1986),  for  example,  discuss  education,  knowledge,  and  research  in 
education  in  terms  of  their  technical  features  and  uses  and  the  ways  these 
features  can  be  about  ethics  and  morality.  And  just  recently,  Feenberg  (1991), 
in  Critical  Theory  of  Technology,  even  speaks  a  little  to  the  relationships  of 
critical  theory,  technology,  and  education,  suggesting  at  one  point  that 
we  could  construct  an  ideal  type  of  a  socialist  economic  code  in 
which  educational  activities,  which  capitalist  society  considers  to 
be  investment  and  evaluates  in  terms  of  productive  efficiency, 
would  be  placed  in  the  category  of  consumption  and  evaluated  as 
contributions  to  welfare  (p.  152). 

However,  in  the  critical  theory  literature,  we  find  virtually  no 
discussion  about  the  combination  of  EBT,  ethics,  and  critical  theory.  The  topics 
are  never  mentioned  in  conjunction  with  one  another.  There  is  no  discussion  of 
which  I  am  aware  about  the  process  relationships  of  these  three  issues.  One  way 
of  putting  this  process  problem  is  to  ask  how  EBT,  morality,  and  critical  theory 
ought  to  be  discussed  and  used  for  some  positive  actions.  And  though  no  critical 
theorist  of  whom  I  am  aware  looks  specifically  at  the  future  of  EBT,  the  critical 
theory  of  Habermas  will  help  to  explain  how  critical  theory  Informs  the  EBT 
problem. 

If  we  ask  about  investigations  of  the  morality  and  ethics  associated  with 
educational  technology,  this  field  is  somewhat  well-represented.  For  instance, 
the  Association  for  Educational  Communications  and  Technology  (AECT)  has  a 
professional  ethics  committee  and  statement,  and  there  have  been  several 
presentations  over  the  last  few  years  at  its  annual  conference  where  ethics, 
morals,  and  technology  were  the  topics  (e.g.,  Damarin,  1990;  Koetting,  1983; 
Nichols,  1988;Schwen,  1988;  Taylor  &  Swartz,  1988). 
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In  terms  of  something  like  EBT  all  by  itself,  just  a  handful  of  people 
(Hanson  and  Gueulette,  1988;  Helnich,  Molenda,  Russell,  1 989;  Knirk  &. 
Gustafson,  1986,  to  name  some  of  the  few)  have  published  In  this  area,  and  In 
most  of  these  Instances  only  tangentlally.  Hanson  and  Gueulette  (1988)  cite 
several  potentially  severe  problems  with  EBT  ("psychotechnologies"  In  their 
terms),  but  they  add,  "What  Is  potentially  new  about  transformational 
technologies  and  truly  Indicative  of  an  authentic  paradigm  shift  Is  the  potential  to 
wed  psychotechnologies  to  an  ecological  vision  which  balances,  not  dominates, 
nature"  (p.  240).  They  urge  us  to  take  responsibility  for  our 
psychotechnologlcal  creations.  They  seem  hopeful  that  we  can  create  a  balanced 
vision.  I,  of  course,  am  not  so  hopeful. 

In  terms  of  EBT  together  with  ethics,  however,  I  can  find  only  a  couple  of 
citations  In  the  last  few  years.  Dede  ( 1 986)  claims  that  "biological 
manipulation  of  humans  alters  Internal  and  affective  capabilities:  emotions, 
neurological  development,  intellectual  capacity,  genetics,  aging"  (p.  43).  And 
though  he  cautions  that  'These  emerging  technologies  may  Impose  more  complex 
ethical  challenges  than  the  atomic  bomb,  computer,  and  television  combined"  (p. 
43),  he  is  generally  optimistic  (very  naively  optimistic,  I  believe)  about  our 
capacity  to  prepare  our  children  to  assume  responsibility  for  the  morality  and 
ethics  of  EBT.  I  have  written  elsewhere  about  EBT  (Nichols,  1990)  and  propose 
that  "EBT  will  cause  profound  doubt,  dilemma,  and  moral  consternation  about 
modern  thinking  and  technology.  What  does  It  mean  to  learn,  to  have  knowledge, 
and  to  be  human"  (p.  138)  In  learning  situations  characterized  by,  say,  an 
educational  black  market  for  memory  enhancing  drugs? 

In  sum,  then,  very  few  people  are  examining  a  future  with  EBT.  One  way 
to  pursue  the  notion  that  EBT  will  proliferate  and  be  destructive  Is  to  turn  to 
philosophers  of  morals  for  guidance.  Moreover,  philosophy  Is  an  approach  which 
seeks  wisdom  and  seeks  to  get  at  basic  beliefs  (Randall  &  Buchler,  1970),  and, 
so,  calls  on  evidence  that  includes  but  goes  beyond  the  highly  rational,  the 
strictly  practical. 

HABERMAS  AND  CRITICAL  THEORY 

So  we  turn  to  the  critical  theorist,  Habermas,  who  analyzes  rationality  in 
relation  to  morals,  and  whose  theory  of  communicative  action  is  critical  of  some 
media  and  of  extreme  rationality.  According  to  Ewert  (1991),  "The  central 
theme  In  Habermas'  work  Is  the  development  of  a  comprehensive  theory  of 
rationality  sufficient  to  encompass  science,  morality,  and  art"  (p.  346) 
[emphasis  mine].  There  are  at  least  three  Issues  on  which  Habermas  is  very 
attractive.  He  speaks  to:  that  In  us  which  Is  less  rationalistic  (often  called  "the 
other"),  the  character  of  meaningful  communicative  acts,  and  morality. 

On  the  first  issue,  he  speaks  to  that  in  humans  which  Is  not  purely 
rational,  but  which  Is  also  tacit,  intuitive,  and  only  Implicit.  He  calls  the 
"place"  where  this  happens  the  "lifeworld"  and  says  that  It  "is  an  implicit 
knowledge  that  can  not  be  represented  In  an  Infinite  number  of  propositions;  It  is 
hollstlcally  structured  knowledge,  the  basic  elements  of  which  intrinsically 
define  one  another;  and  It  is  a  knowledge  that  does  not  stand  at  our  disposition, 
Inasmuch  as  we  can  not  always  make  it  conscious  and  place  It  in  doubt  as  we 
please"  (Habermas,  1981/1984,  p.  336). 
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Through  certain  kinds  of  language,  this  lifeworld  helps  to  influence  our 
affairs.  In  his  theory  of  communicative  action,  Habermas  says  that,  "Language 
and  culture  are  constitutive  of  the  lifeworld  itself"  (Habermas,  1981/1987,  p, 
125).  And  "when  people  are  in  truthful,  right,  or  authentic  communication, 
they  use  language  and  culture  in  such  a  way  that  language  and  culture,  too.  are 
implicit,  are  only  on  the  horizon,  are  part  of  the  reservoir  from  which  people 
select  knowledge"  (Nichols,  1990,  p.  35). 

Habermas  argues  that  there  are  several  kinds  of  knowledge  and  that  the 
communicative  use  of  knowledge  means  competence  In  the  "unconstrained, 
unifying,  consensus-bringing  force  of  argumentative  speech"  (Habermas, 
1981/1984,  p.  10)  in  order  to  transmit  cultural  knowledge,  fulfill  norms 
appropriate  to  a  given  social  context,  and  socialize  Individuals  at  the  level  of 
personality  (Habermas,  1981/1987,  p.  63).  Unlike  posltlvlst  claims,  validity 
claims  In  this  realm  are  made  not  by  logical-Instrumental  uses  of  knowledge  but 
via  "nondeductive  relations  between  pragmatic  units  (speech  acts)"  (Habermas, 
1981/1984,  p.  22). 

But  Habermas  does  not  seek  to  eliminate  the  rational  altogether;  he  wants 
the  lifeworld  and  rationality  to  meet  via  argumentation,  and  he  says,  "The 
rationality  Inherent  in  this  practice  is  seen  In  the  fact  that  communicatively 
achieved  agreement  must  be  based  in  the  end  on  reasons.  And  the  rationality  of 
those  who  participate  in  this  communicative  practice  is  determined  by  whether, 
If  necessary,  they  could,  under  suitable  circumstances,  provide  reasons  for  their 
expressions"  (Habermas,  1981/1984,  p,  17).  "Suitable"  requires  that  there 
be  no  coercion,  but  shared  intersubjectlvlty. 

A  major  concern  for  Habermas  is  that  rationalization  of  the  lifeworld 
changes  communication  and  social  structures,  sometimes  in  deleterious  ways:  we 
rely  more  on  communicative  consensus  that  Is  more  abstract;  pedagogical 
approaches  to  child-rearing  and  formal  schooling  increase;  society  becomes  more 
legalistic  and  bureaucratic,  while  concrete  value-orientations  decrease  and 
actions  aimed  more  at  success  than  at  mutual  understanding  Increase. 

In  these  circumstances,  more  and  more  complex  social  systems  have  to  be 
coordinated,  and  Habermas  calls  some  of  the  mechanisms  which  do  so 
"delinguistified  media."  They  are  delingulstlfied  because  they  require  less  and 
less  direct  consensual  speech/argumentation.  Examples  of  these  media  Include 
prestige,  power,  influence,  and  money  (end  sometimes  electronic  mass  media]. 
They  threaten  us  when  they  either  condense  or  replace  mutual  understanding  in 
language: 

lie  Jia  such  as  money  and  power  attach  to  empirical  ties;  they 
encode  a  purposive-rational  attitude  toward  calculable  amounts  of 
value  and  make  it  possible  to  exert  generalized,  strategic  Influence 
on  decisions  of  other  participants  while  bypassing  processes  of 
consensus-oriented  communication.  Inasmuch  as  they  do  not 
merely  simplify  linguistic  communication,  but  replace  it  with  a 
symbolic  generalization  of  rewards  and  punishments,  the 
lifeworld  contexts  in  which  processes  of  reaching  understanding 
are  always  embedded  are  devalued  in  favor  of  media-steered 
interactions;  the  lifeworld  is  no  longer  needed  for  the  coordination 
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of  action  (Habermas,  1981/1987,  p.  183). 

Further,  these  media  become  more  and  more  complex  so  fewer  people 
understand  or  take  responsibility  for  them,  and  "In  this  way  the  destruction  of 
urban  environments  as  a  result  of  uncontrolled  capitalist  growth,  or  the 
overbureaucratlzatlon  of  the  educational  system,  can  be  explained  as  a  'misuse' 
of  media"  (Habermas,  1981/1987,  p.  293).  Note,  however,  his  claim  that  the 
base  difficulty  with  this  kind  of  history  Is  attributable  ultimately  to  "an  elitist 
splitting  off  of  expert  cultures  from  contexts  of  action  In  dally  life"  (p.  330);  it 
Is  the  Increasingly  rationalistic  ways  In  which  a  few  of  us  think  and  act  that 
cause  us  danger. 

Habermas  (1984)  makes  this  same  argument  with  reference  to  education 
hav<ng  been  shifted  to  schools  and  away  from  parents  and  community.  He  notes 
the  ways  we  try  to  protect  children  In  these  removed  circumstances  by  litigation 
and  rules,  but  he  concludes  that,  "The  protection  of  pupils'  and  parents'  rights 
against  [potentially  harmful  aspects  of]  educational  measures...is  gained  at  the 
cost  of  a  judlclalizatlon  and  bureaucratization  that  penetrate  deep  Into  the 
teaching  and  learning  process"  (p,  371).  Bringing  schooling  under  the  domain  of 
law  abstractly  groups  people  In  educational  competition  and  achievement 
processes.  So  school  law  becomes  the  norm,  without  much  consideration  of 
students'  and  educators'  concerns,  needs,  Interests,  or  life  relationships.  To  the 
extent  this  description  of  rationalization  holds,  It  "has  to  endanger  the 
pedagogical  freedom  and  Initiative  of  the  teacher.  The  compulsion  toward 
litigation-proof  certainty  of  grades  and  the  over-regulation  of  the  curriculum 
lead  to  such  phenomena  as  depersonalization,  Inhibition  of  Innovation,  breakdown 
of  responsibility,  Immobility,  and  so  forth"  (p.  371). 

Habermas  (1 990)  takes  his  theories  about  communicative  action  and  the 
degradation  of  llfeworlds  Into  his  most  explicit  treatment  of  morality,  Morel 
Consciousness  end  Communicative  Action.  In  this  book,  he  gives  his  most 
complete  discussions  about  what  he  means  about  the  nature  of  morals.  He  does  not 
speak  directly  to  the  problem  I  raise  here  (that  EBT  will  proliferate,  be 
dangerous,  and  is  often  Immoral),  but  he  does  offer  a  description  of  moral 
principles  and  processes  associated  with  It.  And  here,  too,  he  argues  that  the  only 
good  way  to  universal  moral  principles  Is  via  argumentative  communication  as 
sketched  above. 

Two  principles,  or  "rules,"  guide  this  discourse  on  morals.  The  first  Is 
what  he  calls  a  universal  principle,  and  It  reads,  "All  affected  can  accept  the 
consequences  and  the  side  effects  Its  general  observance  can  L»e  anticipated  to  have 
for  the  satisfaction  of  everyone's  Interests  (and  these  consequences  are  preferred 
to  those  of  known  alternative  possibilities  for  regulation)"  (p,  65).  Habermas 
tries  to  show  that  this  is  a  universal  because,  for  Instance,  everyone  who 
honestly  enters  an  argument  presupposes  such  "rules"  as  "no  speaker  may  be 
prevented,  by  Internal  or  external  coercion,  from  exercising  his  rights  [of 
speech!"  (p.  89), 

His  second  principle  of  discourse  ethics  Is  derived  from  conditions  of  the 
first  and  says,  "Only  those  norms  can  claim  to  be  valid  that  meet  (or  could  meet) 
with  the  approval  of  all  affected  by  their  capacity  as  participants  In  a  practical 
discourse"  (p.  66),  This  is  a  practical  discourse  In  which  participants  must  not 
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only  reflect  and  vote  on  the  norm  in  question  but  must  partake  of  a  fully 
intersubjectlve  process  "that  Is  reflexive  In  nature;  only  It  can  give 
participants  the  knowledge  that  they  have  collectively  become  convinced  of 
something"  (p.  67).  He  calls  this  type  of  conversation  a  transcendental- 
pragmatic  justification,  in  that  the  intuitive  in  us  and  the  rational  In  us  meet  in 
conversation. 

Habermas  Is  to  be  applauded  for  giving  us  somewhat  of  a  process  and 
universal  guides  to  achieving  consensus.  His  explanation  of  the  features  of 
morals  and  the  processes  by  which  we  might  meet  consensus  Is  very  attractive;  it 
Is  a  way  to  join  the  Intellectual  with  the  tacit  In  us--and  to  do  so  in  conjunction 
with  communities  of  people.  He  shows  how  various  communities  which  fight 
delinquistification  and  ecological  damage  have  emerged  more  or  less  of  their  own 
holistic  volition,  not  by  legislation  and  rationalization.  He  shows  that  there  is 
hope  that  a  moral  condition  among  people,  one  which  is  resistent  to  rampant 
technology,  can  gain  a  foothold.  He  searches  for  ways  too  rationalized  lives  can 
get  back  to  a  more  balanced,  moral  condition. 
HOW  HABERMAS  FAILS  ME 

However,  I  can  not  get  enough  help  from  him  on  my  immediate  need  to 
resist  technology  now.  Habermas  is  clear  that  he  believes  that  the  road  he  is 
suggesting— the  transcendental-pragmatic  justification  of  a  rule  of 
argumentation—is  selective,  and  that  the  rule  "is  not  compatible  with  all 
substantive  legal  and  moral  principles"  (p.  94).  He  says,  further,  the*  the  rule 
"is  a  rule  of  argumentation  only"  (p.  94).  In  other  words,  applying  these 
principles  to  specific  situations  Is,  at  best,  difficult  and,  at  worst,  nearly 
impossible  since  even  agreeing  to  hold  conversations  such  as  he  suggests  Is  often 
nearly  Impossible.  When  people  (men  mostly,  I  suspect)  come  together  to 
discuss  meaningful,  pragmatic  matters,  the  participants  often  are  not  satisfied 
with  others'  arguments.  Think  about  nations  trying  to  find  ways  to  alleviate 
natural  resource  destruction.  At  least  to  this  point  in  history,  participants' 
conversations  are  often  verbal  fist  fights,  not  consensual  conversations.  And 
think  of  most  higher  education  academic  units.  It  is  in  these  units  we  migh* 
expect  the  most  community  of  conversation;  however  It  Is  in  many  of  these  units 
where  some  of  the  least  constructive  conversation  occurs,  at  least  in  several  of 
the  education  departments  and  colleges  of  which  I  am  aware! 

And  as  he  says,  there  is  no  specific  content  attached  to  these  rules,  only  a 
way  to  proceed  (p.  94).  And  though  having  a  procedui  j  is  helpful,  we  also  need  a 
content  to  which  to  attach  these  principles.  For  me,  then,  Habermas  is  the  too 
extreme  rationalist  who  hasn't,  for  the  moment  anyway,  come  up  with  anything 
more  than  too  general  principles  whose  applicability  Is  apparently  next  to 
useless  In  practice! 
RORTY'S  LIBERAL  IRONIST 

Rorty  (1989)  would,  in  general  at  least,  agree  with  my  last 
characterization.  He  criticizes  Habermas  for  thinking  that,  through 
communicative  reason,  a  democratic  society  will  embody  rationalism, 
democracy,  universallsm  (p.  67).  Rorty  thinks  that  we  should  be  willing  to  call 
"true"  anything  which  comes  out  of  free  conversation,  let  alone  communicative 
reason.  He  wants  to  drop  "the  image  of  a  preestablished  harmony  between  the 
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human  subject,  and  the  object  of  knowledge"  (p.  67)  that  are  advanced  by 
Habermas'  rational  speech  situations. 

Whereas  Habermas  sees  a  conversation  which  converges  in  rationality  and 
unlversallsm,  Rorty  wants  a  more  contingent,  historical  and  varied  existence. 
He  wants 

to  stop  asking  for  universal  validity.  I  want  to  see  freely  arrived 
at  agreement  as  agreement  on  how  to  accomplish  common  purposes 
(e.g.,  prediction  and  control  of  behavior  of  atoms  or  people, 
equalizing  life-chances,  decreasing  cruelty),  but  I  want  to  see 
these  common  purooses  against  the  background  of  an  increasing 
sense  of  radlcul  diversity  of  private  purposes,  of  the  radically 
poetic  character  of  Individual  lives,  and  of  the  merely  poetic 
foundations  of  the  'we-consclousness'  which  lies  behind  our  social 
Institutions  (p.  67). 

At  the  base  of  his  argument  Is  the  notion  of  a  "liberal  Ironist."  And  of 

this  notion,  Rorty  says, 

liberals  are  the  people  who  think  that  cruelty  Is  the  worst  thing 
we  do.  I  use  'ironist'  to  name  the  sort  of  person  who  faces  up  to 
the  contingency  of  his  or  her  own  most  central  beliefs  and  deslres- 
-someone  sufficiently  hlstoriclst  and  nominalist  to  have  abandoned 
the  Idea  that  those  central  beliefs  and  desires  refer  back  to 
something  beyond  the  reach  of  time  and  ch&nce  (p.  xv). 
Rorty  has  certain,  sometimes  contrary,  points  of  view  about  moral 

progress.  He  makes  the  point, 

The  view  I  am  offering  says  thut  there  Is  such  a  thing  as  moral 
progress,  and  that  this  progress  Is  Indeed  In  the  direction  of 
greater  human  solidarity.  But  that  solidarity  Is  not  thought  of  as 
recognition  of  a  core  self,  the  human  essence,  in  all  human  beings. 
Rather,  it  Is  thought  of  as  the  ability  to  see  more  and  more 
traditional  differences  (of  tribe,  religion,  race,  customs,  and  the 
like)  as  unimpc.iant  when  compared  with  similarities  with 
respect  to  pain  and  humiliation— the  ability  to  think  of  people 
wildly  different  from  ourselves  as  included  In  the  range  of  'us.' 
That  Is  why  I  said...that  detailed  descriptions  of  particular 
varieties  of  pain  and  humiliation  (In,  e.g.,  novels  or 
ethnographies),  rather  than  philosophical  or  religious  treatises, 
were  the  modern  Intellectual's  principal  contributions  to 
progress  (p.  192). 

One  thing  that's  nice  about  Rorty's  view  is  that  our  apparently  endless 
search  for  a  universal,  principled  moral  existence  does  not  lead  necessarily  to  an 
empty  or  confused  condition.  Though  no  a  priori  law  exists  to  tell  us  how  to  Hv© — 
or  how  to  avoid  technological  calamity— we  can  take  comfort  In  our  similar 
searches  to  avoid  personal  pain  and  humiliation.  To  this  mundane  but  solid  extent 
can  we  make  some  moral  progress  regarding  technology. 
THE  TROUBLE  WITH  RORTY'S  VIEW 

But  there  are  several  ways  In  which  Rorty's  position,  too,  isn't  very 
helpful  In  my  search  for  a  way  we  might  agree  that  EBT  is  always  dangerous— as 
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well  as  anything  else  it  might  be.  For  one,  his  conversations  simply  don't  address 
technology  Itself  enough.  Also,  he  treads  dangerous  ground  when  and  If  he  thinks 
that  being  able  to  predict  and  control  people  will  lead  to  equality  and  decreased 
cruelty.  In  one  way,  I'm  making  the  argument  that  technology  Is  control  and 
predictability,  and  this  condition  Is  what  Is  most  dangerous.  I  agree  with  Ellul 
(1990)  that  with  control  always  comes  some  degradation  or  another  of  humans. 
Rorty  does  not  fully  address  this  degradation  very  well. 

And  also,  there  is  a  good  argument  that  his  positions  lead  to  unfettered, 
unhelpful  relativism.  One  could  extend  (and  many  have!)  his  reasoning  to  moke 
the  point  that  whatever  anyone  decides  at  any  one  moment  Is  acceptable.  I  don't 
want  us  giving  fetal  brain  tissue  from  deceased  babies  to  pre-schoolers  who  are 
otherwise  perfectly  ready  for  learning,  even  If  an  entire  community,  city,  or 
state  have  agreed  to  do  so.  I  want  to  know  the  extend  of  the  negotiations,  of  the 
moral  considerations,  of  the  Interests  represented  by  actors  In  the  negotiations. 
Mostly,  we  need  guiding  over-riding  moral  principles  because  not  everything 
nations  and  states  agree  on  Is  moral. 

But  what  most  bothers  me  about  his  philosophy  is  that  his  stences  are  all 
stated  in  positive,  propositional  language,  and  so  they  seem  to  contradict 
themselves  by  being  suspiciously  sure  of  what  they  say.  How  is  it  sensible  that 
he  can  say,  on  one  hand,  that  life  Is  contingent,  then  say,  on  the  other  hand, 
something  so  sure  as, 

our  responsibilities  to  others  constitute  only  the  public  side  of 
our  lives,  a  side  which  competes  with  our  private  affections  and 
our  private  attempts  at  self-creation,  and  which  has  no  automatic 
priority  over  such  private  motives.  Whether  It  has  priority  in 
any  given  case  Is  a  matter  for  deliberation,  a  process  which  will 
usually  not  be  aided  by  appeal  to  'classical  first  principles.' 
Moral  obligation  is...to  be  thrown  in  with  a  lot  of  other 
considerations,  rather  than  automatically  trump  them  (p.  194). 
How  can  he  be  so  sure  of  this  statement  when  he  has  just  said  that  there  is 
contingency  In  everything?  If  his  statement  about  no  automatic  trumping  is  so, 
how  can  he  begin  to  claim  so  surely  that  pain  and  humiliation  are  the  central 
human  concerns? 

And  lastly,  he  doesn't  give  me  guidance  by  showing  me  particular  enough 
morality  or  ethics.  His  advice  is  to  create  a  morality  that  fits  the  times  and  could 
be  useful,  but  I'm  looking  for  something  more  concrete.  Do  I  have  to  wait, 
probably  until  It's  too  late,  for  all  of  us  to  get  together  and  reach  agreement  about 
dangerous  technology?  That  agreement  may  never  happen,  and  I  want  to  know 
soon  what  widely  accepted  moral  principles  to  go  by  In  this  matter. 
BARRETT'S  WILL  TO  FAITH 

To  this  point,  then,  my  hunt  for  moral  guidance  In  an  age  of  blo- 
technologlcal  schooling  has  been  unsure.  Habermas  can  not  sustain  his  arguments 
for  a  moral  principle  on  which  we  can  all  agree,  and  Rorty  leaves  open  the  door 
for  relativism  that  might  have  us  embracing  technology  too  easily,  and  he  gives 
me  little  hope  for  explicit  principles  that  help  us  to  avoid  self-destruction  via 
EBT. 

William  Barrett  (1987),  on  the  other  hand,  focuses  not  just  on  questions 
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of  human  rationality,  argumentation,  or  shared  pain  and  humiliation  as  a  common 
human  basis.  He  appears  to  look  farther  and  higher.  He  looks  at  moral  questions 
by  raising  our  moral  will  to  the  highest  of  conditions  with  which  to  be  graced  as  a 
human  being.  He  believes  that  to  achieve  freedom  we  must  "try  to  restore  the 
moral  will  to  a  central  and  primary  role  In  the  human  personality"  (p.  254). 

One  of  his  main  themes  Is  that  faith  leading  to  freedom  and  action  Is 
stymied  by  technique.  Because  we  live  predominantly  In  a  technical  era,  where 
Instead  of  the  moral  will  we  are  the  "will  to  self-assertion  and  dominance;  the 
will  to  power"  (p.  253),  we  live  in  dangerous  times,  and  'The  question  for  our 
generat1on...1s  whether  or  not  mankind  will  decide  for  liberty  or  sink  under  some 
modern  form  of  tyranny.  And  this  question  Is  In  the  end  a  moral  matter"  (p. 
246). 

He  suggests  that  we  can  sometimes  succeed  In  our  moral  responsibilities 
by  using  traditional  and  "ritual  practices  or  codes  of  behavior  that  might 
possibility  prepare  the  will  for  deliverance"  (p.  263).  He  says,  'To  surrender 
yourself  In  the  ritual  act  Is  a  gesture  of  your  humility  and  Ignorance, 
helplessness  and  hopefulness,  of  which  the  epistemologlcal  act  is  quite  incapable" 
(p.  309).  I  take  Barrett  to  mean  that  prayer,  thanksgiving,  and 
acknowledgement  of  the  human  will  to  live  are  moral  rituals. 

Barrett  has  given  us  some  clues  about  being  in  a  moral  state.  He  suggests 
actions  for  getting  to  this  moral  condition:  we  must  have  faith,  we  must  attend  to 
the  spiritual  In  us,  we  should  not  believe  too  much  in  technique  and  let  it  rule 
our  existences. 

THE  TROUBLE  WITH  BARRETT 

And  I  believe  Barrett  Is  correct,  as  far  as  he  goes.  There  Is  at  least  one 
way  In  which  I  must  point  out  what  might  be  a  contradiction.  Yes,  people  rletirly 
are  filled  /vith  a  will  to  power  (see  studies  by  Michel  Foucault  on  this  Issue,  of 
course.)  We  are  filled  with  the  self  will  to  dominate.  And  Barrett  (1978)  seems 
to  argue  that  to  be  in  a  more  moral  and  a  less  controlling  condition  requires  that 
we  must  will  to  be  will-less?  But  isn't  this  a  contradictory  action?  We  should 
will  to  be  will-less?  He  attemps  to  answer  this,  at  least  In  terms  of  a  blance 
between  the  known  In  life  and  the  mystery  of  life,  with  the  response  "At  every 
moment  of  every  day  of  our  lives,  we  are  Involved  In  a  continuous  and  living 
transaction  with  the  unknown.  This  being  the  case,  what  then  shall  be  my 
strategy?  It  is  this:  to  multiply  my  points  of  contact  with  this  unknown 
wherever  I  can  let  It  work  to  my  advantage;  to  deny  myself  none  of  its  privileges 
that  its  unbounded  resources  might  o\ ,er"  (p.  307).  I  think  Barrett  says  we  are 
both  conditions  and  that  faith  Is  that  which  we  already  possess  by  virtue  of 
existence;  humans  can  not  be  in  one  state  exclusive  of  the  other,  In  a  faithful 
condition  but  not  in  a  will-to-power  condition.  Rather,  he  would  say  that  we  tend 
toward  one  condition  or  another  and  that  we  should  now  tend  toward  the  moral  and 
the  possibilities  for  greater  faithfulness. 

Instead,  perhaps  we  gain  moralUy  by  giving  up  power  to  the  greatest 
extent  possible,  by  letting  go  of  actions  and  thoughts  that  attach  to  willfulness.  It 
is  an  interesting  question:  do  we  gain  a  good  life  by  giving  up  to  the  extent 
possible  our  tendencies  to  dominate  existence,  or  do  we  achieve  the  good  life 
through  a  more  assiduous,  conscious,  purposeful  willing  of  faith? 
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Likewise,  I  want  suggest  an  ironic  contradiction  that  begs  for  attention 
(though  this  is  not  the  most  appropriate  forum  for  the  bulk  of  that  attention)  in 
this  discussion  and  asks  in  what  sense  can  or  should  we  resist  or  deny  our 
rationality,  our  techno-centrism  if  you  will?  It  is  not  an  aside  to  ask  if  anything 
should  be  done.  That  is,  should  we  assume  the  power  to  change  things  by  thinking 
and  resisting?  I  do  not  think  so,  but  perhaps  we  should  just  let  nature,  including 
our  human  nature,  take  its  course?  Shouldn't  we  stop  trying  to  control  existence 
when  it's  our  controlling  aspect  that  may  be  the  culprit  to  begin  with?  We  might 
end  in  some  kind  of  oblivion  if  we  just  let  it  go  on,  but  we  might  end  up  in  a  worse 
kind  of  oblivion  (psychic,  ecological,  or  cultural  oblivion)  I've  speculated  about 
elsewhere  (1990)  If  we  pursue  answers  these  problems.  This  Is  quite  an  uneasy 
dilemma  we  live  Inl  Damned  If  we  do  will  our  existence,  less  damned  If  we  don't? 
SOME  IMPLICATIONS 

Few  people  are  examining  the  question  of  EBT.  So  I  predict  that,  as  is 
historically  typical  of  us,  we  will  wait  until  the  worst  aspects  of  educational  and 
general  technology  are  upon  us  before  we  act  to  fully  consider  their  Implications. 
In  the  relatively  short-term  future,  the  number  of  EBTs  will  Increase 
surprisingly  quickly;  we  will  be  faced  with  Incredibly  wider-ranging  student 
and  social  differences,  so  cultural  upheavals  will  occur  (Nichols,  1990). 
Habermas  notes  that  any  moral  can  be  valid  only  when  It  has  the  approval  of  all 
affected,  and  elitist  cultures  splitting  off  from  mainstream  cultures  cause  losses 
of  the  llfeworld  and  the  physical  world.  I  expect  that  the  uses  of  EBTs  will  not  be 
determined  In  consensual  fashion,  but  will  be  driven  by  a  certain  elite  few.  And 
some  groups  of  people,  the  disadvantaged,  will  not  stand  by  calmly  as  other 
groups  use  technology  to  maintain  or  widen  any  socio-economic  and  ecological 
advantages.  I  claim  that  technology  and  education  will  produce  soclo-cultural 
wars. 

These  are  not  especially  unusual  predictions.  A  more  unsettling 
revelation  comes  to  me  from  somewhat  beyond  the  text  I  have  overtly  presented 
so  far,  end  that  revelation  makes  It  difficult  for  me  to  be  faithful  In  this  search 
for  moral  wisdom.  Having  looked  at  each  author's  thoughts,  I  conclude  that 
Habermas,  Rorty,  and  Barrett  amount  to  next  to  nothing  In  the  current 
mainstream  conversations  about  today's  education,  technology,  and  goodness. 
Most  people  In  colleges  of  education,  to  say  nothing  of  people  In  our  other  schools, 
have  not  heard  of  these  three  essayists.  That  their  names  are  unfamiliar  Is 
excusable.  Bui  th?ir  Ideas  go  largely  unnoticed  In  the  world  of  day-to-day 
American  schooling  and  Its  reform.  We  In  education  are  dealing  mostly  with 
issues  of  the  obviously  practical  and  the  here-and-now:  how  can  technology 
improve  student  performance,  how  can  we  produce  more  technically  competent 
teachers,  how  can  American  schools  keep  up  with  the  Japanese? 

So  I  repeat,  how  do  we  raise  the  philosophical-moral  issues  to  more 
conscious  and  pragmatic  levels,  especially  for  teachers  and  learners?  Doing  so  is 
extremely  difficult  when,  as  Strike  (1991)  sees  It,  the  problem  Is  "che 
Inability  of  the  public  to  consume  sophisticated  [philosophical]  discussions  about 
public  life"  (p.  414).  As  Rorty  (1989)  puts  It,  ho  v  do  we  foster  an 
"increasing  sense  of  radical  diversity  and  of  private  purposes"  that  will  foster 
greater  human  solidarity? 
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One  way  to  make  the  issues  more  publicly  prominent  is  to  increasingly 
raise  these  issues  in  public  forums.  We  should  continue  to  raise  doubt  about 
human  technical  aspects.  Habermas'  communicative  reason,  Rorty's  liberal 
ironist,  and  Barrett's  will  to  faith  are  ideas  that  can  become  popular,  perhaps 
even  before  too  many  self-Inflicted  disasters  enlighten  us.  I  think  the  main 
question  about  the  Ideas  Is  when  we  will  address  them— while  we  have  a  little 
time  or  when  we  are  at  an  Intellectual,  physical,  spiritual,  and  emotional  rock 
bottom.  I  suspect  we'll  wait  for  the  bottom. 

Still  though,  we  should  move  toward  a  greater  degree  of  fundamental 
thinking  now,  and  I  believe  that  the  best  practical  faculty  we  have  for 
encouraging  the  questions  Is  personal  conversation.  This  Is  the  case  because 
conversation  Is  a  main  way  In  which  the  Ideas  mentioned  herein  have  the  most 
likelihood  of  being  meaningful  to  us  (also  see  Ong,  1982,  on  this  issue).  And  in 
Its  favor,  the  critical  theory  of  Habermas  as  described  In  his  theory  of 
communicative  action  (1981/1984;  1981/1987)  is  of  some  help  to  us.  (It  Is 
an  Important  aside  to  note  that  personal  conversation  will  have  the  benefit  of 
encouraging  democracy.) 

In  order  to  stop  the  bur  .aucratlzation  and  technologizing  of  education,  we 
must  take  this  conversation,  this  democracy,  more  directly  to  teachers  and 
learners  than  some  of  us  In  higher  education  have  done  heretofore.  They  do  not 
need  more  highly  abstract,  academic  papers.  Learners  and  teachers  do  not  need 
more  coercing  from  technology  companies  or  from  principals  and  parents  blinded 
by  the  light  of  technology.  They  need  radical,  on-site  support  from  those  of  us 
who  are  fortunate  enough  to  have  the  professorships  and  other  such  positions  that 
allow  us  time,  some  good  Ideas,  and  the  desire  to  mobilize  teacher  and  learner 
resistance,  where  It  is  necessary.  We  should  join  and  participate  in  the  NEA,  for 
instance. 

In  my  own  oarticipation,  I  must  look  closely  at  my  motives  (Nichols, 
1987);  I  must  constantly  ask  if  my  actions,  my  will,  are  the  will  to  have 
destructive  and  dominating  power  for  myself  and  others  or  the  will  to  let  ,n  the 
faith  and  conversation  In  life  that  exists  already.  I  look  for  change  based  on  my 
attending  to  my  intuition  that  much  technology  Is  needlessly  dangerous,  and  by 
going  to  teachers  and  students  with  these  ideas;  I  will  not  extol  the  purported 
virtue  of  technology  disengaged  from  its  moral/existential  base.  This  is 
especially  the  case  in  classes  I  teach,  but  It  Is  the  case  everywhere  I  am. 

And,  regardless  of  our  inability  to  pin  down  many  specific  universal 
moral  principles  in  terms  of  Habermas,  Rorty,  or  Barrett,  I  believe  th  sre  are 
universals— at  least  ones  that  go  on  for  most  people  for  long,  long  times.  We 
should,  for  instance,  follow  the  universal  that  we  must  press  the  search  for 
moral  conditions,  moral  progress,  and  see  to  It  that  this  progress  is  indeed  in  the 
direction  of  greater  human  solidarity.  Searching  in  and  of  itself  is  a  moral  act  if 
It  is  guided  by  a  desire  to  Improve  our  circumstances  In  the  whole  of  existence. 

Another  meaningful  morality  is  the  Inclination  to  live  rather  than  die. 
This  inclination  appears  to  me  to  be  Inherent  in  mostly  every  animal  and  person, 
and  so  It  is  essentially  a  universal.  Becker  (1973)  has  written  wonderfully  on 
this  issue  and  shows  that  most  often  people  try  to  live.  Barrett  (1987)  goes  so 
far  as  to  say,  "Death  remains  the  supreme  adventure  to  test  the  moral  will"  (p. 
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260).  Life,  then,  is  unavoidably  a  question  of  morality,  whether  we  acknowledge 
so  or  perpetuate  and  exacerbate  the  dangers  of  technology  by  staying  unaware  of 
the  questions. 
CONCLUSION 

To  end  then,  most  of  us  live  the  moral  life  simply  by  clinging  to  life.  And 
Ironically,  technology  is  a  manifestation  of  this  clinging,  a  manifestation  which 
Is  outgrowing  its  usefulness/faithfulness.  As  Barrett  suggests,  maybe  we  need  to 
tend  to  the  more  existential  or  spiritual  aspect  of  our  lives. 

And  the  morality  of  living  Is  an  Idea  from  which  many  other  morals  can  be 
said  to  derive.  "Treat  your  neighbor  as  yourself,"  for  example.  Put  Ironically 
again,  there  seems  to  be  a  contrasting,  Ironic  human  universal  that  says  we 
should  use  (and  extend?)  our  conscious,  rational  capacities.  If  we  can  think  of 
something  technological,  we  will  make  it.  No  matter  how  destructive  It  may  be. 
(And,  of  course,  some  technologies  could  be  positive.) 

About  this  irony  of  life  versus  over-extended  technology,  there  Is  another 
moral  Idea  that  might  be  helpful:  We  must  el  least  achieve  a  balanced  use  of 
technology  (as  is  the  topic  of  Koyaanisqatsi,  Reggio,  1983),  so  that  we  don't  have 
schools,  people,  life  out  of  moral  and  physical  balance,  so  that  we  might  avoid  too 
many  unnecessary  and  intentional  deaths. 

Perhaps  ultimately,  as  P.D.James  (1991)  suggests,  "Conscience  is  the 
voice  of  God."  However  you  describe  your  god. 
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Abstract 

The  purpose  of  this  study  was  to  evaluate  the  design 
and  implementation  of  10  neuropathology  interactive 
videodisc  instructional  (IVI)  modules  used  by  Michigan 
State  University  medical  students  in  the  College  of 
Osteopathic  Medicine  and  the  College  of  Human 
Medicine.  The  evaluation  strategy  incorporated  c  mixed 
method  approach  using  qualitative  and  quantitative  data  to 
examine  levels  of  student  acceptance  for  the  modules; 
ways  in  which  IVI  modules  accommodate  different  learner 
styles;  and  to  what  extent  the  modules  facilitate  the  at- 
tainment of  higher  level  learning  objectives.  Students 
rated  the  units  highly  for  learning  effectiveness;  many  stu- 
dents reported  group  interaction  as  beneficial;  and  students 
expressed  a  desire  for  more  IVI  in  the  curriculum.  The 
paper  concludes  with  recommendations  for  future  use  of 
interactive  videodisc  tectmology  in  the  teaching/learning 
process. 

Introduction 

The  1984  report  issued  bv  the  Panel  on  the  General 
Professional  Education  of  the  Physician  and  College 
Preparation  for  Medicine  (GPEP)  challenged  medical  edu- 
cators to  make  substantive  changes  in  both  content  and  de- 
livery of  medical  school  curricula.^  Especially  relevant 
to  this  study  is  the  GPEP  recommendation  that  medical 
educators  implement  alternative  teaching  methods  so  as  to 
reduce  students1  reliance  on  lectures  as  a  primary  source 
for  it.  ;?rmation.[2]  Also  relevant  are  report  recommenda- 
tions that  medical  schools  and  faculty:  (1)  adopt  teaching 
methods  for  students  who  have  the  ability  to  learn  inde- 
pendently and  provide  opportunities  for  further  develop- 
ment of  this  skill;  (2)  reduce  passive  teaming  and  require 
students  to  be  active,  independent  learners  and  problem 
solvers;  and,  (3)  promote  the  effective  use  and  application 
of  information  science  and  computer  technology.^  The 
MSU  Colleges  of  Osteopathic  and  Human  Medicine  have 
instituted  a  curricular  revision  process  to  incorporate  the 
GPEP  recommendations. 

A  partial  answer  to  successful  implementation  of  the 
GPEP  recommendations  has  reportedly  been  found  in  the 
instructional  application  of  computer  based  interactive 
videodisc  technology. ^] [5]  Authors  Xakellis  and  Gjerde 
state  that:  "Computer  aided  instruction  is  currently 
viewed  by  many  as  one  approach  to  meeting  this  educa- 
tional need'1  for  increasing  problem  solving  and  self  study 
skills.^]  Among  medical  subjects  that  have  used  interac- 
tive videodisc  instruction  (IVI)  in  instructional  programs 
are  anatomy,  radiology,  pathology,  and  psychia- 
try.[4],[7],[8]  jn  studyt  interactive  videodisc  units 
were  an  integral  pari  of  preclinical  neuropathology  instruc- 
tion. 


Interactive  videodisc  instruction,  as  defined  here,  is 
the  use  of  a  computer  based  instructional  program  that  ac- 
cesses visual  images  stored  on  a  videodisc  in  an  order  con- 
trolled by  the  learner.  A  programmed  computer  accesses 
different  sections  of  the  videodisc  or  instructional  text,  de- 
pending on  the  viewer's  response  to  various  alternatives. 
This  instructionJ  delivery  strategy  provides  individual- 
ized, self  paced  content  information  accompanied  by  in- 
stantaneous feedback.  Since  the  medium  is  inherently  vi- 
sual, it  is  especially  well  suited  to  any  discipline  requiring 
motion,  animation,  pictures,  or  other  visual  stimuli  to  fa- 
cilitate the  learning  process.  Michigan  State  University 
personnel  developed  the  IVI  neuropathology  modules  in 
HyperCard  for  Macintosh  computers  using  images  from 
the  "Slice  of  Life"  videodisc  (a  generic  videodisc  dis- 
tributed by  the  University  of  Utah). [9]  These  modules 
were  used  by  undergraduate  medical  students  in  the 
Michigan  State  University  College  of  Osteopathic 
Medicine  (MSU/COM)  and  College  of  Human  Medicine 
(MSU/CHM), 

This  study  incorporates  elements  of  field  based, 
ethnographic  and  other  qualitative  evaluation  strategies 
with  traditional  quantitative  methods.  These  multiple 
methods  were  implemented  in  an  attempt  to  generate  a 
body  of  relevant  learner  response  criteria  that  might  be  ap- 
plicable to  other  technology  based  learning  settings.  The 
evaluation  strategy  was  a  conscious  attempt  to,  as  Hagler 
suggests,  "move  to  studies  of  instruction  and  learners,  not 
media."[l®]  The  evaluation  process  was  patterned  on  what 
Patton  describes  as  the  "qualitative-naturalistic-formative 
approach...especially  appropriate  for  programs  that  are  de- 
veloping, innovative,  or  changing,  where  the  focus  is  on 
program  improve  ment,  facilitating  more  effective  imple- 
mentation, and  exploring  a  variety  of  effects  on  partici- 
pants.'^1  1]  This  research  process  seeks  to  address  the  lim- 
itations and  inadequacies  inherent  in  research  and  evalua- 
tion studies  which  use  experimental  and  quasi  experimen- 
tal designs  to  compare  interactive  videodisc  technology 
with  other  instructional  delivery  systems  or  "traditional 
instruction,"  often  with  inconclusive  results.t^],[13] 

Because  a  qualitative  inquiry  approach  was  included, 
some  data  gathering  strategies  were  generated  by  the  eval- 
uation process  itself.  For  example,  early  results  from  on- 
line quantitative  evaluation  data  allowed  the  implementa- 
tion team  to  synthesize  a  series  of  questions  for  focus 
group  interviews.  Both  focus  group  interviews  and  on- 
line data  lead  to  three  major  questions  for  the  implementa- 
tion team:  (1)  What  are  the  levels  of  student  acceptance 
for  the  implemented  IVI  materials?;  (2)  Did  the  IVI  mod- 
ules accommodate  different  learner  styles?;  and  (3)  To 
what  extent  do  IVI  modules  facilitate  the  attainment  of 
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higher  level  course  objectives?  These  questions  will  be 
addressed  in  the  Discussion  section  of  this  paper. 

Method 

IVI  Implementation  •  Interactive  videodisc  instruc- 
tional modules  were  developed  on  10  neuropathology  top- 
ics; ncurocy tology/  cytopathology,  degenerative  disorders, 
increased  intracranial  pressure,  inflammatory  diseases, 
slow  vims  diseases,  neoplasms,  traumatic  disorders,  cere- 
brovascular disorders,  developmental  disorders,  and  toxic/ 
metabolic/nutritional  disorders.  Each  IVI  module  consists 
of:  a  five  question  pretest  with  an  indication  of  correct 
and  incorrect  responses;  an  instructional  lesson  with  re- 
lated review  questions  covering  the  content  required  for  ex- 
aminations; a  posttest  with  content  feedback  for  each  an- 
swer; case  studies  designed  to  reinforce  concepts  presented 
elsewhere  and  support  problem  solving  strategies  related 
to  neuroanatomy,  neuropathology  and  neurology;  and  a 
glossary  accessed  from  the  lesson  or  case  study,  or  used 
separately.  A  randomly  accessible  table  of  contents 
(index)  is  available  in  the  lesson  to  permit  student  control 
over  lesson  sequencing.  A  "comments11  field  is  provided 
for  students  to  leave  notes  to  the  instructor  or  instruc- 
tional designers.  An  electronic  mail  system  is  being 
tested  to  expedite  student-instructor  dialogue  regarding 
questions  as  they  arise. 

There  are  three  curricular  formats  in  the  two  medical 
schools  at  Michigan  State  University  and  the  neu- 
ropathology IVI  modules  were  implemented  according  to 
the  nature  of  each  curriculum.  Described  below  are  the 
three  types  of  curricula  and  modes  of  implementation. 

1 .  The  College  of  Osteopathic  Medicine  teaches  neu- 
ropathology within  a  systems  curriculum.  The  IVI 
neuropathology  modules  were  utilized  two  ways  in 
different  years. 

•  During  Fall  1989, 118  third  year  students  enrolled  in 
the  neuroscience  system  course  were  awarded  extra 
credit  points  (one  per  module)  for  viewing  and  evalu- 
ating the  eight  IVI  modules  then  available.  The 
modules  included:  neurocytology/cytopathology;  de- 
generative disorders;  increased  intracranial  pressure; 
inflammatory  diseases;  slow  virus  diseases;  neo- 
plasms; traumatic  disorders;  and  cerebrovascular  disor- 
ders. During  Fall  1990,  a  more  extensive  evaluation 
and  tracking  system  was  available.  The  112  third 
year  students  enrolled  in  the  neuroscience  system 
course  were  required  to  view  three  IVI  modules:  neo- 
plasms; cerebrovascular  disorders;  and  a  student  se- 
lected module.  Computer  use  was  monitored  for  re- 
quired modules.  The  remaining  IVI  modules  were 
recommended,  but  us?  was  elective. 

2.  The  College  of  Human  Medicine  uses  a  two  track  cur- 
riculum: 

•  Students  in  Track  I  are  taught  in  discipline  based 
courses.  During  Spring  1990,  68  second  year  stu- 
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dents  enrolled  in  the  neuropathology  course  were  re- 
quired to  view  four  IVI  modules  (neoplasms;  cere- 
brovascular disorders;  trauma;  inflammatory  disorders) 
with  computer  use  monitored.  The  remaining  I\  I 
modules  were  recommended,  but  use  was  elective. 
Lectures  and  written  class  materials  were  also  avail- 
able, as  in  previous  years. 

•  Students  in  Track  II  are  taught  in  a  problem  based, 
independent  study  curriculum  where  "focal  problems" 
organize  basic  science  content  During  Spring  1990, 
42  first  year  students  were  enrolled  in  the  "Impaired 
Consciousness"  focal  problem  where  the  primary  con- 
tent is  neuroscience.  The  following  IVI  modules 
were  on  the  list  of  required  course  materials:  neo- 
plasms; cerebrovascular  disorders;  inflammatory  dis- 
orders; trauma;  and  increased  intracranial  pressure. 
However,  computer  use  was  elective. 

The  data,  therefore,  span  the  period  from  Fall  1989  to 
Fall  1990  and  incorporate  information  about  340  MSU 
medical  students  enrolled  in  three  curricular  formats. 

Two  on-line  questionnaires  -  a  module-specific  evalu- 
ation and  a  student  information  form  -  were  used  to  collect 
initial  quantitative  data.  A  computerized  student  tracking 
system  provided  important  support  data  to  more  fully  ana- 
lyze usage,  time  spent  in  the  module  components,  and 
levels  of  acceptance  for  the  IVI  course  materials. 

IVI  Module-Specific  Evaluation  •  Questions  pre- 
sented after  each  module  assessed  the  effectiveness  of  vari- 
ous instructional  attributes  of  each  module  component; 
e.g.,  lesson,  case  study,  posttest  question*,  and  glossary. 
An  open  comments  Held  was  available  to  record  student 
comments  about  other  features  of  the  module  not  included 
above. 

Student  Information  Form  -  A  student  information 
form  collected  data  about  prior  computer  experience,  effec- 
tiveness of  group  versus  individual  use,  optimal  group 
size,  appeal  of  overall  design  features,  and  the  preferred 
role  of  computer  instruction  in  the  medical  school  curricu- 
lum. 

Student  Tracking  System  -  A  computerized  student 
tracking  system  collected  data  about  the  amount  of  time 
spent  in  each  individual  module  component  by  each  stu- 
dent, module-specific  pretest  and  posttest  scores  for  correct 
first  responses,  and  the  group  size  for  the  reported  session. 

Focus  Groups  *  During  the  term  the  course  was  offered 
to  CHM  students,  focus  groups  were  organized.  Both 
Track  I  and  Track  II  students  were  invited;  however,  due  to 
schedule  constraints,  Track  II  students  did  not  participate. 
There  were  5  to  10  Track  I  students  in  each  of  three  focus 
group  sessions.  To  encourage  voluntary  participation, 
each  meeting  was  scheduled  at  noon  with  lunch  provided. 
The  session  format  included  a  brief  explanation  of  the 
background  and  rationale  for  developing  ti  e  IVI  modules, 
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orientation  to  the  purpose  of  the  focus  group,  and  a  de- 
scription of  the  evaluation  study  prior  to  discussion  of  the 
interview  guide  questions.  The  interview  guide  explored 
the  following  areas:  hardware,  utility  of  the  IVI  modules 
to  help  students  achieve  levels  of  mastery,  appropriateness 
of  module  content,  organizational  factors,  motivational 
factors,  and  self-testing  components  within  the  module. 

Results 

Data  from  the  Module-Specific  Evaluation 
Form  -  Table  1  presents  a  summary  of  the  data  obtained 
for  selected  questions  from  the  on-line  evaluation  forms 


for  all  students.  Each  of  the  340  students  had  an  opportu- 
nity to  fill  out  evaluation  forms  on  from  five  to  ten  mod- 
ules, depending  on  their  auricular  format.  A  total  of  547 
forms  were  analyzed;  however,  not  all  questions  were  an- 
swered. The  total  number  of  responses  for  each  question 
is  indicated  in  the  table.  Bartlett's  Test  for  Homogeneity 
of  Variance  across  the  three  curricular  formats  indicates  no 
significant  differences  in  response  variability^among  the 
groups  (p=.60).  The  three  groups  were:  (1)  fcOM  1990 
students;  (2)  CHM  Track  I  students;  and  (3)  CHM  Track  n 
students.  Therefore,  responses  were  pooled  for  each  ques- 
tion. 


Table  1.    Results  on  Module-Specific  Evaluation  Forms 

Statement 

LESSON: 

This  unit  helped  me  learn  the  material. 

The  content  presented  was  too  easy. 

The  inclusion  of  videodisc  images  was  valuable. 

More  videodisc  images  should  be  presented. 

The  review  questions  did  not  help  and  should  be  deleted. 

The  computer  diagrams  helped  to  illustrate  concepts. 

I  would  recommend  this  section  to  others. 
CASE  STUDY: 

The  case  study  challenged  me  to  think  about  clinical/pathological  problem  solving. 
The  questions/feedback  effectively  reinforced  neuroanatomy  and  neuropathology. 
There  should  be  more  case  studies  on  this  topic. 
I  would  recommend  this  section  to  others. 
POSTTEST: 

The  explanations  in  the  posttest  helped  to  reinforce  the  main  concepts. 
The  questions  were  too  easy. 
GLOSSARY: 

The  glossary  was  NOT  a  useful  feature. 

1  Number  of  responses  to  each  question. 

^  Mean  of  responses  where  Ustrongly  disagree,  2=disagree,  3=agree,  4=strongly  agree 


% 


Mean^ 

A/SA 

D/J 

542 

3.44 

100 

0 

535 

1.95 

8 

92 

538 

3.45 

98 

2 

534 

3.09 

80 

20 

533 

1.55 

3 

97 

527 

3.29 

98 

2 

534 

3.45 

99 

1 

480 

3.35 

96 

4 

516 

3.38 

98 

2 

478 

2.81 

69 

31 

511 

3.36 

98 

2 

516 

3.44 

98 

2 

521 

2.07 

18 

82 

451 

2.23 

26 

74 

Results  indicate  that  students  rated  all  aspects  of  the 
modules  as  very  valuable  in  helping  master  the  material, 
including  content,  images,  diagrams,  glossary,  and  review 
questions.  In  addition,  many  of  the  open  ended  comments 
written  by  students  were  extremely  positive  and  empha- 
sized the  value  and  effectiveness  of  the  IVI  modules  in 
helping  them  master  course  content. 

Data  from  the  Student  Information  Form  - 

Completion  of  this  form  was  totally  optional.  Although 
the  sample  was  relatively  small  (n=47)  and  self  selected, 
the  researchers  believe  that  the  opinions  expressed  were 
important  for  planning  and  future  program  development. 
Results  indicate  a  fairly  even  distribution  in  previous 
computer  experience,  ranging  from  no  previous  experience 
to  experienced  users.  The  majority  of  students  (66%)  re- 
porting prior  experience  had  used  a  Macintosh  computer. 
A  majority  of  students  (72%)  who  used  the  modules  in 
small  group  study  (two  to  three  students)  stated  that  inter- 
action was  beneficial.  In  response  to  a  question  about  op- 
timal study  group  size,  49%  stated  that  two  was  optimal; 
and  36%  said  three  was  optimal.  The  remaining  students, 


only  15%,  preferred  more  than  three  in  a  study  group. 
Most  students  (91%)  liked  the  option  of  leaving  a  com- 
ment and  87%  requested  an  on-line  mechanism  to  obtain 
timely  answers  to  their  questions  from  the  instructor. 
Responding  to  the  statement,  "Computer  instruction  is 
best  suited  for  replacing  lectures  for  some  material,11 67% 
of  the  students  agreed  and  33%  disagreed.  Students  said 
they  would  prefer  to  spend  more  time  using  computer 
based  instruction  (CBI)  than  they  are  currently  spending 
(e.g.,  38%  are  currently  spending  three  hours  or  more  per 
week;  72%  said  they  would  prefer  to  use  CBI  three  hours 
or  more  per  week).  When  asked  about  additional  features, 
more  students  requested  an  expansion  of  the  case  study 
section  than  any  of  the  other  parts. 

Data  from  the  Computerized  Student  Tracking 
System  -  Table  2  provides  a  summary  of  the  tracking 
data  important  for  the  analysis  of  IVI  module  usage.  Data 
were  recorded  for  each  individual  student,  whether  working 
individually  or  in  a  small  group.  The  different  modes  of 
implementation  provide  the  opportunity  to  analyze  student 
use  in  different  ways  so  student  tracking  data  were  recorded 
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separately  for  each  curricular  format  and  mode  of  imple- 
mentation. For  all  curricular  formats,  students  using  the 
modules  spent  enough  time  to  master  the  material  in  both 
the  lesson  and  case  study.  There  was  no  substantial  differ- 
ence in  the  time  spent  in  each  section  based  on  whether 
use  was  required  or  not  required;  the  amount  of  content 
appeared  to  be  the  major  factor  in  determining  average 
time  spent. 

(Insert  Table  2  about  here) 

•  For  Track  I  students  (discipline-based  curricu- 
lum), four  modules  were  required.  Analysis  of  the 
number  of  students  logged  on  to  optional  modules  in- 
dicates that  many  students  (about  half  of  those  en- 
rolled) used  non-required  IVI  modules,  although  the 
content  was  also  covered  in  lecture  and  written  course 
materials. 

•  For  Track  II  students  (problem-based  curricu- 
lum), content  in  five  of  the  modules  was  included  in 
the  exam,  but  use  of  the  computer  was  not  required. 
Student  tracking  data  indicate  that  a  majority  of  stu- 
dents did  use  the  IVI  modules.  For  modules  with  the 
major  neuropathology  content,  Track  II  students  spent 
more  time  in  the  lesson  than  Track  I  students.  Since 
Track  II  students  have  less  opportunity  to  see  exam- 
ples of  visual  images  of  neuropathology  lesions  pre- 
sented, the  increased  time  in  the  lesson  may  be  at- 
tributed to  the  increased  time  students  spent  interact- 
ing with  the  visual  material. 

•  For  College  of  Osteopathic  Medicine  students  in  the 
neuroscience  system  during  Fall  of  1989,  a  total  of 
798  extra  credit  points  were  awarded  to  the  1 18  stu- 
dents enrolled  (85%  of  the  total  possible  points  avail- 
able for  this  activity).  Eighty-seven  students  (74%) 
completed  all  eight  modules  available  at  that  time  and 
only  nine  students  did  not  use  any  of  the  modules. 
Although  students  did  not  have  to  complete  the  Case 
Study  to  receive  extra  credit  points,  about  half  of  the 
students  used  this  section. 

•  For  College  of  Osteopathic  Medicine  students  in  the 
neuroscience  system  during  Fall  1990,  students  were 
required  to  complete  three  IVI  modules:  cerebrovascu- 
lar disorders;  neoplasms;  and  one  more  of  their 
choice.  About  one  third  of  the  students  used  the  non- 
required  modules. 

Data  from  the  Student  Focus  Group  Sessions  - 
The  following  six  points  summarize  the  three  focus  group 
sessions: 

1.  Hardware:  Most  students  stated  that  the  equipment 
was  reasonably  easy  to  use.  Levels  of  computer  liter- 
acy varied  greatly;  however,  lack  of  familiarity  with 
hardware  did  not  seem  to  be  an  inhibiting  factor  in 
students1  perceived  comfort  with  the  computers. 
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2.  Utility  of  the  IVI  modules  to  help  students  achieve 
levels  of  mastery:  A  majority  of  students  remarked 
that  the  modules  were  best  suited  for  acquiring  basic 
concepts  and  principles,  and  practicing  with  case  stud- 
ies. Several  students  recommended  that  the  course  be 
structured  to  make  the  modules  required  prior  to  at- 
tending lecture.  Assuming  that  students  used  the 
modules  to  understand  the  basics,  lectures  could  then 
be  used  for  higher  order  learning,  such  as  integration, 
clinical  correlations,  and  problem  solving.  In  this 
way,  lecture  time  could  be  spent  going  beyond  the 
module  acquired  basics  to  discussion  of  application  of 
the  basic  content 

3.  Content  and  design  of  the  computer  based  modules: 
Helpful  features  specifically  mentioned  included  the 
glossary  and  the  visual  ability  to  compare  normal  and 
abnormal  images.  Recommendations  for  improve- 
ment included  providing  text  references  with  the  feed- 
back for  the  sample  test  questions  and  tying  the  neu- 
ropathology IVI  module  content  to  previously  learned 
neuroanatomy  course  material. 

4.  Organizational  factors:  Several  students  commented 
on  the  excellent  organization  of  the  IVI  teaching  mod- 
ules and  how  well  they  fit  into  the  overall  course. 
For  example,  students  said  they  liked  the  reference 
points  provided  throughout  a  given  module,  signaling 
how  far  along  they  were  in  the  lesson,  e.g.,  card  15  of 
75.  Students  expressed  appreciation  for  the  variety 
and  diversity  in  case  studies  and  the  parallel  between 
the  IVI  modules  and  the  course  syllabus.  Students 
preferred  units  completed  in  one  sitting  and  judged 
certain  modules  to  be  too  long. 

5.  Motivational  factors:  Several  students  mentioned  that 
requiring  completion  of  certain  modules  was  an  effec- 
tive strategy,  since  it  introduced  students  to  the  com- 
puter based  modules.  The  majority  of  focus  group 
participants  indicated  that  they  went  on  to  complete 
more  than  the  required  modules  because  they  judged 
them  to  be  useful  and  fun.  Without  the  requirement, 
students  speculated  that  many  would  not  have  tried 
the  IVI  modules  at  all.  Some  students  commented 
that  they  were  motivated  by  the  novelty  of  using 
computer  based  instruction  as  an  alternative  to  tradi- 
tional learning  methods. 

6.  Self  examination:  Students  commented  frequently 
and  spontaneously  on  the  evaluation  aspects  of  the 
modules  and  on  the  pretests  and  posttests  in  particu- 
lar. They  liked  the  idea  of  the  self-tests  but  had  sev- 
eral suggestions  for  improvement  Among  the  most 
frequent  suggestions  were:  increase  the  number  and 
frequency  of  test  questions  throughout  the  lesson  and 
make  the  questions  more  challenging  (e.g.,  more  syn- 
thesis or  analysis  level  questions). 
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Student  Tracking  Data 


CO 


CHMTrackl 


CHM  Track  II 


COM  1989 


COM  1990 


Module 

L 

n 

C 

minutes 
/student 
L  C 

L 

n 

C 

minutes 
/student 
L  C 

L 

n 

C 

minutes 
/student 
L  C 

L 

n 

C 

minutes 
/student 
L  C 

#  *  Cerebrovascular 

32 

29 

53 

22 

21 

14 

65 

25 

76 

40 

55 

15 

69 

50 

60 

30 

#*  Neoplasms 

47 

40 

65 

21 

20 

11 

71 

20 

68 

35 

51 

23 

67 

51 

80 

20 

*  Trauma 

55 

44 

45 

13 

20 

16 

56 

25 

85 

45 

39 

11 

34 

11 

44 

11 

*  Inflammatory 

22 

21 

51 

22 

16 

12 

63 

16 

85 

33 

42 

8 

26 

16 

45 

16 

Increased  I  CP 

16 

12 

34 

13 

19 

13 

56 

13 

84 

41 

26 

19 

11 

8 

51 

12 

Cytopathology 

19 

NA 

33 

NA 

NA 

NA 

NA 

NA 

56 

NA 

25 

NA 

21 

NA 

29 

NA 

Degenerative 

34 

31 

56 

7 

NA 

NA 

NA 

NA 

95 

54 

33 

6 

25 

13 

44 

11 

Developmental 

38 

35 

63 

9 

NA 

NA 

NA 

NA 

NA 

NA 

NA 

NA 

22 

7 

45 

4 

Slow  Virus 

21 

20 

20 

6 

NA 

NA 

NA 

NA 

92 

51 

25 

5 

39 

20 

17 

7 

Toxic-Met-Nutr 

30 

22 

49 

14 

NA 

NA 

NA 

NA 

NA 

NA 

NA 

NA 

25 

7 

38 

11 

#  Required  for  COM 

L 

=  Lesson 

C 

=  Case  Study 

NA  =  Not  Applicable 

(519 


9 
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Discussion 

Data  emerging  from  the  various  evaluation  method- 
ologies  support  a  high  level  of  student  acceptance  for  the 
implemented  IVI  materials.  Student  tracking  data  suggest 
that  students  took  the  time  in  each  module  to  cover  the 
material  and  explore  various  module  alternatives  rather 
than  doing  the  minimum  needed  to  receive  credit. 
Module-specific  evaluation  forms  reflect  (his  positive  re- 
sponse with  100%  of  students  stating  that  they  would  rec- 
ommend the  modules  to  other  students  and  that  the  IVI 
lesson  helped  them  learn  the  material.  Focus  group  re- 
sponses and  student  tracking  data  reinforced  high  levels  of 
acceptance,  with  students  reporting  they  they  completed 
optional  modules  and  that  they  liked  the  IVI  approach. 
The  high  level  of  student  acceptance  was  related  in  part  to 
a  high  correlation  of  IVI  module  content  with  course  syl- 
labus and  lecture  content. 

One  objective  in  module  development  was  to  accom- 
modate different  learner  styles.  Students  who  used  the  IVI 
modules  reported  different  patterns  of  use  of  the  learner 
control  options:  pacing  control,  sequence  control,  content 
control,  and  whether  to  view  the  materials  in  groups  or 
individually.  Students  commented  that  the  modules  satis- 
fied a  variety  of  objectives  including:  as  an  advance  orga- 
nizer to  acquire  basic  concepts;  as  a  supplement  to  the  lec- 
ture; for  review;  and  in  some  instances  as  a  replacement 
for  lecture.  Students  strongly  supported  the  use  of  real 
pathology  images  on  IVI  to  supplement  text  information 
and/or  reinforce  learning.  Thus  different  styles  of  learning 
and  different  abilities  of  students  were  accommodated  by 
the  IVI  design  and  implementation.  This  was  a  conscious 
effort  by  the  instructional  designers. 

The  attainment  of  higher  level  learning  is  a  continu- 
ous goal  in  medical  education.  Students  reported  that 
when  modules  were  viewed  prior  to  class,  the  time  spent 
in  lecture  was  used  to  synthesize  information  rather  than 
merely  obtain  facts.  Students  expressed  approval  for  us- 
ing the  case  studies  to  attain  higher  level  course  objec- 
tives, e.g.,  to  practice  clinical  problem  solving  based  on 
patient  history,  integrate  and  apply  basic  coi^cpts  and 
principles,  and  integrate  and  synthesize  previously  learned 
material. 

In  summary,  the  multiple  methods  approach  used  in 
this  study  generated  a  body  of  relevant  learner  response 
data  consistent  with  previous  process  study  outcomes. 
Students  demonstrated  a  high  level  of  acceptance  for  the 
neuropathology  IVI  materials,  different  learner  styles  were 
accommodated  by  the  design  of  the  modules,  and  higher 
level  objectives  were  facilitated  by  the  design  and  imple- 
mentation of  the  IVI  modules.  Thus,  this  study  supports 
the  increased  application  of  interactive  videodisc  technol- 
ogy in  the  medical  school  curriculum  to  achieve  the  rec- 
ommendations outlined  in  the  GPEP  Report.  Successful 
IVI  curricular  integration  must  attend  to  issues  that  in- 
volve careful  planning,  monitoring,  and  follow  up. 


Specific  Recommendations 

1 .  Appropriate  use  of  IVI  should  be  L  egrated  within  &*? 
curriculum,  not  merely  provided  as  an  adjunct  to 
learning. 

2.  IVI  material  should  relate  to  content  on  exams  and 
should  incorporate  feedback  to  the  student 

3.  IVI  modules  should  incorporate  learner  control  op- 
tions that  accommodate  various  learner  styles. 

4.  Initial  exposure  to  IVI  modules  should  be  strongly 
encouraged  or  required 

5.  Group  interactions  (2-3  students/group)  should  be  fa- 
cilitated 

6.  Students9  perspectives  on  instructional  issues  should 
be  solicited  in  order  to  design  IVI  units  and  plan  im- 
plementation strategies  to  best  meet  the  needs  of  the 
target  student  population. 
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Introduction 

As  someone  trained  as  a  filmmaker,  I  often  wonder  how  the  years  spent  learning 
and  practicing  my  craft  have  changed  me.  In  recent  years,  I  have  become 
increasingly  aware  of  instances  when  I  find  myself  slipping  into  what  I  think  of  as 
my  "filmmaker  mode".  As  I  pour  over  research  articles  in  an  attempt  to  see  inter- 
relationships and  connections,  revise  a  paper  on  my  word  processor,  or  pare  down 
and  organize  data  on  a  spreadsheet,  I  am  reminded  of  the  many  hours  I  have  spent 
in  an  editing  room  sifting  through  reels  of  film  to  find  just  the  right  combination  of 
shots.  I  am  perhaps  most  aware  of  this  "filmmaker  mode"  when  I  am  in  a  new 
place.  I  consciously  begin  to  select  a  series  of  mental  snapshots  and  to  differentiate 
between  the  many  layers  of  sound  present  around  me.  In  essence,  I  break  my 
surroundings  down  into  discrete  chunks  of  visual  and  aural  data  which  later  helps 
me  remember  what  I  have  seen  and  heard  in  rich  detail.  These  personal  anecdotes 
suggest  that  the  practice  of  filmmaking  has  changed  me  -  that  it  has  in  some  small 
way  altered  the  way  I  perceive  and  interact  with  the  world  around  me. 

Sol  Worth  (1973)  was  the  first  to  note  the  potential  cognitive  effects  of  filmmaking 
and  suggested  that  the  practice  of  filmmaking  could  be  harnessed  as  an  educative 
experience  in  the  classroom.  Worth  described  the  predominant  use  of  film  in  the 
classroom  --  film  as  a  delivery  system  -  as  a  "substitute"  for  books  and  lectures. 
He  further  characterized  the  intent  of  this  use  of  the  medium  as  "learning  through 
film"  ~  what  the  student  is  intended  to  learn  is  the  content  presented  in  the  film. 
The  second  and,  according  to  Worth,  the  most  powerful  use  of  film  in  the 
classroom,  relates  to  how  the  medium  works  as  a  system  of  communication  - 
something  Worth  called  "learning  about  media".  Worth  suggested  that  the  skills 
developed  through  such  an  understanding,  a  set  of  skills  referred  to  as  "film 
literacy",  would  allow  the  learner  to  view  the  world  in  a  new  way: 

Filmmaking  could  become  one  of  the  important  tools  by  which  we  allow 
and  help  the  child  as  well  as  the  adult  to  develop  skill  in  building  cognitive 
structures  and  in  structuring  reality  in  a  creative,  communicative  way. 
Although  we  can  teach  through  film  [emphasis  added],  we  must  begin  to 
understand  how  the  structure  of  film  itself  and  the  visual  modes,  in  general, 
structure  our  ways  of  organizing  experience,  (p.  302) 

Theoretical  Support 

The  notion  that  the  act  of  filmmaking  may  change  the  filmmaker  in  important  ways 
is,  in  part,  explained  by  a  body  of  theoretical  work  that  describes  ways  humans  are 
able  to  perceive  and  represent  the  world  in  a  variety  of  symbolic  forms  or  systems 
(Olson  and  Bruner,  1974;  Salomon,  1979;  Gardner,  1983).  This  work  makes  two 
basic  claims.  The  first  claim  is  that  cognitive  representation  and  processing  are 
carried  out  in  various  symbolic  modes.  The  second  is  that  these  cognitive 
operations  can  be  influenced  by  the  symbol  systems  employed  by  media  (Clark  and 
Salomon,  1986).  For  example,  the  act  of  writing  a  cogent  essay  draws  upon 
different  symbolic  competencies  than  the  act  of  composing  a  symphony  or  taking  a 
photograph,  because  each  of  these  mediated  messages  is  cast  in  a  different  set  of 
symbolic  forms.  (Gardner,  1983). 

One  of  the  most  useful  "symbol  system"  theories  is  one  proposed  by  Salomon 
(1979)  because  Salomon  makes  an  important  link  between  symbol  systems  and 
media  (Clark  and  Salomon,  1986).  Salomon's  theory  suggests  that  both  media  and 
the  human  mind  use  symbols  to  represent,  store,  and  manipulate  information  and 
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that  some  of  the  symbol  systems  used  in  cognition  are  cultivated  through  the 
manipulation  of  the  symbol  systems  employed  by  media. 

Each  symbolic  competence  is  comprised  of  a  set  of  cognitive  skills  that  must  be 
mastered  in  order  for  an  individual  to  be  fluent  in  that  particular  symbol  system. 
Researchers  like  Gardner  (1983)  and  Olson  (1974)  prefer  to  think  of  these  various 
kinds  of  symbolic  competencies  as  different  forms  of  intelligence.  In  fact,  Gardner 
identifies  five  different  kinds  of  intelligence  or  symbolic  competency:  linguistic 
intelligence,  musical  intelligence,  logical-mathematical  intelligence,  spatial 
intelligence,  and  bodily-kinesthetic  intelligence.  What  cognitive  skills  might  the  act 
of  filmmaking  cultivate?  Prior  empirical  work  (Tidhar,  1984;  Sutton-Smith,  1979) 
and  a  detailed  examination  of  the  cognitive  tasks  involved  in  filmmaking  suggests 
that  among  the  most  likely  kinds  of  cognitive  abilities  to  be  developed  from  the 
practice  of  filmmaking  are  spatial  abilities  and  skills  of  abstract  reasoning. 

Filmmaking:  A  Conceptual  Analysis 

Filmmaking  is  a  difficult  process  to  examine  because  it  involves  many  diverse  tasks 
and  is  often  carried  out  by  many  skilled  craftspersons  such  as  directors,  actors, 
camerapersons,  and  editors.  This  is  particularly  true  within  the  professional  feature 
film  industry  where  it  is  difficult  to  describe  any  one  individual  as  a  filmmaker. 
The  vast  majority  of  films  made,  however,  are  not  feature  films  and  are  not 
characterized  by  such  a  large  and  diverse  labor  force.  These  films  are  often  made 
by  individuals  who  are  responsible  for  all  phases  of  production  from  initial  concept 
through  shooting  and  editing  to  finished  film.  Under  such  conditions,  the  term 
"filmmaker"  more  accurately  describes  the  people  involved. 

In  its  basic  form,  filmmaking  can  be  seen  as  two  fundamental  processes:  the 
collection  and  crafting  of  discrete  visual  and  aural  elements  into  communication  -  a 
series  of  messages  that  conveys  meaning  (Eisenstein,  1949).  In  film  parlance,  the 
collection  stage  is  referred  to  as  "filming"  or  "cinematography".  The  crafting  stage 
is  called  "editing".  Both  stages  are  particularly  complex  because  of  the  cognitive 
visualization  requirements  of  the  tasks.  It  is  important  to  note  that  such  a  definition 
suggests  that  filmmaking  need  not  necessarily  occur  with  conventional  motion 
picture  cameras  and  editing  equipment,  The  sort  of  collecting  and  crafting  of 
images  and  sound  described  here  can  be  carried  out  with  video  equipment,  with  a 
sti1!  camera  and  an  audio  cassette  recorder,  or  even  with  the  latest  in  microcomputer 
technology. 

During  the  collection  stage  the  filmmaker  must  carefully  visualize  the  way  a  finished 
shot  will  look.  This  includes  not  only  visualizing  the  content  of  the  shot:  the 
setting,  the  time  of  day,  season,  the  movement  of  objects  within  the  frame,  but  also 
the  relationship  of  the  frame  to  this  visual  content.  Should  the  frame  emphasize 
certain  objects  and  if  so  how?  For  example,  should  the  camera  shoot  the  scene 
from  a  high  or  low  angle?  Should  the  camera  shoot  the  scene  from  the  front  or 
back  or  perhaps  from  the  side?  Each  of  these  options  must  be  arefully  imagined. 

The  filmmakers  visualization  task  is  made  more  complex  by  the  fact  that  the  camera 
and  scene  being  shot  may  not  be  static.  For  example,  the  camera  may  move 
towards  an  actor  as  the  actor  simultaneously  moves  towards  the  camera  or  the 
camera  may  "crane"  into  the  air  as  an  actor  moves  away  from  the  camera.  This  sort 
of  complex  visualization  requires  the  ability  to  mentally  picture  an  object  or  objects 
from  any  conceivable  vantage  point.  In  the  simplest  case,  both  the  vantage  point 
and  the  object  are  static.  In  the  most  complex  case,  both  the  vantage  point  and  the 
object  are  moving  independently  of  each  other. 
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The  crafting  phase  of  filmmaking,  often  referred  to  as  editing,  entails  the  assembly 
of  individual  shots  and  corresponding  sound  (Pudovkin,  1958).  It  is  during  this 
stage  of  production  that  the  raw  data  are  reviewed  and  distilled  into  the  filmmaker's 
original  conception.  It  is  during  the  assemble  stage  that  a  filmmaker's  visualization 
ability  is  tested  once  again.  The  filmmaker  must  organize  the  shots  in  such  a  way 
as  to  ensure  that  each  edit  appears  invisible.  This  is  done  by  making  sure  that  the 
finished  scene  creates  the  illusion  of  a  continuous  flow  of  action.  This  means  that 
the  many  takes  of  a  scene  that  were  shot  sequentially  must  be  edited  to  appear  as  if 
they  were  shot  at  the  same  time.  To  create  such  a  continuity  of  action,  the 
filmmaker  must  be  able  to  visualize  the  finished  scene  from  beginning  to  end  as  it 
will  appear  in  the  finished  movie.  The  visualization  task  called  upon  at  this  stage  is 
similar  to  the  one  required  at  the  collection  stage.  The  filmmaker  must  have  the 
ability  to  imagine  the  objects  in  the  scene  as  they  move  relative  to  each  other  over 
time.  In  addition,  the  filmmaker  must  be  able  to  visualize  these  changes  from 
different  vantage  points. 

An  actual  editing  problem  illustrates  the  complexity  of  the  visualization  task.  An 
editor  must  edit  together  four  different  shots  of  a  car  smashing  into  a 
passenger  train.  Shot  one  shows  a  long  shot  of  a  car  hurtling  towards  a  level 
crossing  as  a  high  speed  passenger  train  approaches  in  the  background.  Shot  two 
is  taken  from  the  point  of  view  of  the  driver  of  the  car.  We  see  the  driver's  hands 
thrown  up  in  fear  as  the  train  suddenly  appears  through  the  windshield.  Shot  three 
is  a  close  up  of  the  driver  as  he  shields  his  face.  Shot  four  is  a  long  shot  of  the  car 
crashing  into  the  train  and  exploding  in  a  huge  ball  of  fire.  In  order  for  these  shots 
to  flow  smoothly  together,  the  spatial  relationship  between  the  car,  the  train,  and  the 
actor  must  be  the  same  from  edit  to  edit.  The  editor  must  be  able  to  visualize  these 
constantly  changing  spatial  relationships  as  he  or  she  imagines  the  finished  scene. 

Another  important  cognitive  skill  is  Colled  upon  during  the  editing  process.  In  the 
early  stages,  film  editing  is,  essentially,  an  exercise  in  visual  organization.  Images 
must  be  identified  and  then  organized  into  a  workable  conceptual  framework.  This 
organizing  phase,  in  essence,  becomes  an  exercise  in  visual  concept  classification, 
Suitable  categories  must  be  formulated  and  then  each  set  of  images  (a  shot)  must  be 
classified  according  to  the  framework. 

For  example,  shots  may  be  classified  according  to  the  type  of  camera  movement 
used,  the  type  of  coverage  (long  shot,  close-up,  medium  shot),  the  type  of  action 
within  the  shot,  the  location  of  the  shot  or  simply  the  overall  quality  of  the  shot. 
Once  a  framework  has  been  created  and  the  raw  footage  broken  down  and 
organized,  the  filmmaker  begins  to  assemble  the  individual  shots  into  scenes.  Since 
the  ability  to  organize  visual  data  is  apparently  linked  most  with  editing,  while  the 
ability  to  visualize  spatial  forms  is  called  upon  in  both  editing  and  cinematography, 
it  would  seem  likely  that  skill  in  spatial  visualization  is  more  likely  to  be  cultivated 
through  an  initial  filmmaking  experience. 

A  Review  of  Past  Studies 

Several  studies  have  specifically  investigated  the  effect  of  filmm.'ung  on  the 
cultivation  of  cognitive  skills.  Tidhar  (1984)  demonstrated  that  junior  high  school 
students  who  were  taught  filmmaking  skills  showed  significant  improvement  on 
measures  of  logical  inference,  spatial  aptitude,  and  conceptual  focusing  as 
compared  to  a  non-filmmaking  control  group. 

Eighty-seven  fifth  grade  students  were  randomly  assigned  to  four  different  55  hour 
filmmaking  courses.  The  first  group  was  taught  photography  only,  the  second 
script  writing  and  photography,  the  third  photography  and  editing,  and  the  forth 
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was  taught  editing  only.  The  course  lasted  10  weeks  and  consisted  of  two  3  hour 
sessions  per  week.  A  series  of  8  tests,  including  the  DAT  (1980)  space  relations 
and  abstract  reasoning  tests  were  administered  to  the  students  3  v  ;eks  after  the 
completion  of  the  film  course.  Through  factor  analysis,  the  8  scores  were 
combined  into  measures  of  logical  inference,  spatial  aptitude  and  conceptual 
focusing.  All  filmmaking  treatments  and  particularly  the  editing  group,  resulted  in 
significant  increases  in  scores  as  compared  to  a  control  group  which  did  not  receive 
the  treatment. 

A  three  year  study  that  investigated  the  New  York  City  Young  Filmmakers  Program 
(Sutton-Smith,  1979;  Sutton-Smith,  Eadie,  and  Griffin,  1983)  provided  evidence 
that  filmmaking  had  a  significant  educational  impact  upon  children.  One  hundred 
and  fifty  students  ages  5  to  16  years  and  of  varying  ethnic  and  socio-economic 
status  participated  in  the  study.  Approximately  300  short  films  were  produced. 
The  major  hypothesis  of  the  study  was  that  children  who  received  intense  training 
and  practice  in  filmmaking  would  demonstrate  significant  gains  in  their  ability  to 
organize  and  manage  internal  visual  imagery. 

Impact  was  assessed  along  three  dimensions:  perceptual,  cognitive  and  personal. 
Perceptual  competencies  were  defined  in  terms  of  the  learner!s  ability  to  notice 
visual  details  and  to  discern  hidden  aspects  of  reality  which  may  not  have  been  so 
obvious  beforehand.  The  Witkin's  Embedded  Figures  Test  (1954)  and  the 
Goodenough-Harris  Draw  a  Man  Test  (1961)  were  used  as  measures  of  perceptual 
competence.  Cognitive  organization  was  measured  by  performance  on  the 
Wechsler  Intelligence  Scale  for  Children  (WISC)  (1950).  In  particular,  the  sections 
dealing  with  picture  arrangement/picture  completion  and  block  designs  were 
emphasized.  Personal  development  was  equated  with  creativity  which  was 
measured  by  performance  on  the  figural  section  of  the  Torrance  creative  thinking 
test  and  the  individual's  sense  off  personal  autonomy  which  was  measured  by 
performance  on  a  locus  of  control  test. 

Results  of  the  study  showed  that  significant  changes  in  cognitive  measures  were 
directly  related  to  the  number  of  films  made  and  the  amount  of  time  spent  at  the 
workshop.  Those  completing  at  least  one  film  showed  significant  increases  in  their 
performance  on  the  Block  Design  subtest  of  the  WISC  and  a  significant  internal 
shift  on  the  locus  of  control  measure.  In  addition,  a  majority  of  the  children  who 
participated  in  the  filmmaking  workshop  showed  significant  gains  in  scores  on  the 
Embedded  Figures  Test.  Another  important  finding  was  related  to  the 
characteristics  of  learners  who  made  at  least  one  film  compared  with  those  who 
dropped  out.  Those  who  completed  one  or  more  films  had  significantly  higher 
scores  on  the  WISC  Picture  Completion  subtest  and  the  Object  Assembly  subtest; 
had  a  better  memory  for  films  that  they  had  observed  being  made  in  New  York;  and 
were  more  often  the  youngest  of  a  small  family  or  the  eldest  of  a  large  family. 

Both  the  Tidhar  (1983)  and  the  Sutton-Smith  (1979)  studies  suggested  that  the 
cultivation  of  spatial  and  abstract  reasoning  skills  were  among  the  important 
cognitive  benefits  from  prolonged  filmmaking  practice.  Both  studies  examined  the 
cognitive  effects  o"  filmmaking  on  relatively  young  subjects  -  between  the  ages  of 
6-16.  Two  important  weaknesses  of  these  studies  should  be  noted.  First,  both 
studies  relied  on  relatively  weak  treatments  -  approximately  50  to  75  hours  of 
filmmaking  practice.  Second,  both  studies  failed  to  develop  a  compelling  rationale 
for  the  cognitive  measures  used. 
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Spatial  Skills  and  Abstract  Reasoning 

Spatial  ability  (also  referred  to  in  the  literature  as  spatial  intelligence)  is  "skill  in 
representing,  transforming,  generating,  and  recalling  symbolic,  non  linguistic 
information  (Linn  and  Petersen,  1985,  p  1482).  Linn  and  Petersen  (1985)  contend 
that  spatial  ability  is  comprised  of  three  basic  skills:  spatial  perception,  mental 
rotation,  and  spatial  visualization. 

Research  studies  have  shown  that  success  in  certain  academic  subject  areas  and 
vocations  is,  in  part,  a  function  of  an  individual's  spatial  aptitude.  Spatial  scores 
are  directly  related  to  performance  in  courses  such  as  plane  geometry,  drawing, 
mathematics,  and  shop  (McGee,  1979).  Spatial  aptitude  has  also  been  shown  to  be 
a  good  predictor  of  success  in  such  occupations  as  architecture,  computer  science, 
art  and  engineering  (McGee,  1979). 

The  present  study  focuses  upon  one  of  the  most  complex  spatial  abilities,  spatial 
visualization.  Spatial  visualization  tasks  involve  a  number  of  steps  and  must  be 
solved  by  using  "an  analytic  combination  of  visual  and  non-visual  strategies" 
(Thomson,  1989,  p.  17).  Spatial  visualization  subsume  spatial  perception  and 
mental  rotation.  Spatial  perception  involves  the  "comprehension  of  the  arrangement 
of  elements  within  a  visual  stimulus  pattern  and  the  ability  to  remain  unconfused  by 
the  changing  orientation  in  which  a  spatial  configuration  may  be  presented" 
(McGee,  1979,  p.  893).  Mental  rotation  is  the  ability  to  mentally  manipulate  the 
orientation  of  spatial  forms. 

Successful  spatial  visualization  requires  subjects  to  carefully  analyze  a  problem 
visually  and  select  an  appropriate  solution  strategy.  It  is  the  multi-step,  multi- 
strategy,  analytic  characteristic  of  the  task  which  distinguishes  spatial  visualization 
from  other  spatial  skills  (Linn  and  Petersen,  1985). 

Au:*ract  reasoning  is  the  ability  to  determine  relationships  between  visually 
presented,  non-semantic  information  (Anastasi,  1982,  p.  290).  Formalized  tasks  of 
abstract  reasoning  require  the  ability  to  analyze  a  set  of  apparently  disparate 
patterns,  create  a  meaningful  context  for  the  patterns,  and  then  select  like  patterns 
for  inclusion  in  the  0et  (Cronbach,  1960). 

The  ability  to  create  a  meaningful  context  for  visual  information  appears  to  be 
important  during  the  editing  stage  of  filmmaking.  Editing  entails  the  shaping  and 
structuring  of  vast  amounts  of  visual  data  in  the  form  of  hundreds  or  even 
thousands  of  individual  shots.  The  filmmaker  needs  a  coherent  conceptual 
framework  to  aid  in  the  organization  of  the  many  shots. 

Rationale  For  Study 

Determining  whether  spatial  skills  and  skills  of  abstract  reasoning  are  cultivated 
through  filmmaking  is  important  for  four  reasons.First,  the  study  may  yield  support 
for  or  against  the  inclusion  of  filmmaking  within  high  school  and  university 
curricula.  There  are  currently  more  than  3000  film  courses  at  600  universities  and 
colleges  across  North  America  (Bohn  and  Stromgren,  1987).  Since  the  vast 
majority  of  students  enrolled  in  film  production  courses  never  pursue  a  professional 
filmmaking  career,  it  may  be  possible  to  identify  the  development  of  certain 
cognitive  skills  as  a  valuable  by-product  of  making  movies  within  an  educational 
setting.  In  addition,  the  rapid  proliferation  of  inexpensive  home  video  equipment 
has  meant  that  filmmaking  Is  now  an  activity  that  millions  engage  in.  How  might 
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these  individuals  expect  to  be  affected  by  their  hobby?  Several  studies  already 
suggest  that  filmmaking  practice  is  beneficial  by  showing  that  it  cultivates  certain 
cognitive  skills.  For  example,  Tidhar  (1983)  showed  that  a  55  hour  filmmaking 
course  led  to  significant  increases  on  measures  of  logical  inference,  spatial  ability, 
and  conceptual  focusing  among  junior  high  school  students. 

Second,  the  skills  gleaned  from  making  films  may  be  applied  to  apparently  more 
complex  or  more  abstract  activities  -  such  as  essay  writing  (Constanzo,  1985).  In 
other  words,  filmmaking  can  be  employed  as  an  instructional  intervention.  A 
knowledge  of  the  skills  filmmaking  cultivates  will  help  educators  to  design  such 
interventions.  Filmmaking  has  been  used  as  a  means  of  facilitating  high  school 
students'  appreciation  and  understanding  of  literary  structure  (Cox,  1985). 
Filmmaking  has  also  proved  to  be  a  useful  means  of  helping  students  engage  more 
enthusiastically  in  the  revision  of  written  work  (Primeau,  1974).  Since  revision 
(editing)  is  such  a  self-evident  part  of  filmmaking,  students  exposed  to  it  seem  more 
willing  to  rework  other  forms  of  self-expression  such  as  term  papers  and  essays. 
Also,  the  production  of  a  short  film  on  the  American  Civil  War,  for  example,  may 
be  a  more  effective  means  of  offering  certain  students  the  opportunity  to  gather  and 
synthesize  knowledge  about  the  topic  than  writing  a  conventional  term  paper  would 
(Cox,  1985). 

Given  that  filmmaking  can  serve  as  an  instructional  intervention,  it  is  conceivable 
that  aspects  of  filmmaking  practice  could  be  incorporated  as  part  of  a  computer- 
based  lesson  or  data-base  management  program.  For  example,  the  film-editing 
process  could  be  simulated  as  part  of  a  lesson  designed  to  teach  learners  about 
dramatic  structure  in  short  stories.  Learners  would  be  able  to  manipulate  (reorder, 
shorten,  lengthen)  sections  of  a  short  story  and  then  view  the  result  -  just  as  a  film 
editor  would  cut  a  short  film.  Computer-based  data  base  management  programs 
might  be  successfully  manipulated  along  a  film-editing  model.  Information  on  a 
particular  topic  could  be  stored  as  a  specific  shot  and  then  reworked  into  scenes  that 
incorporate  a  number  of  specific  entries.  Other  educational  applications  of 
filmmaking  can  be  identified  and  developed. 

Third,  the  study  may  lead  to  a  better  understanding  of  the  filmmaking  process  itself 
which  in  turn  may  suggest  new  ways  of  teaching  filmmaking.  If  certain  cognitive 
skills  are  fundamental  to  the  making  of  good  films  then  perhaps  these  skills  need  to 
be  stressed  during  any  filmmaking  course. 

Fourth,  the  perspectives  and  methods  used  to  understand  and  harness  the  practice 
of  filmmaking  as  an  educational  activity  may  be  successfully  employed  with  other 
types  of  mediated  practice  such  as  computer  programming  and  data-base  creation 
and  management.  In  other  words,  viewing  the  activity  of  filmmaking  as  an 
educative  experience,  redefines  the  way  the  field  of  educational  technology  views 
the  role  of  media  in  education.  It  suggests  that  media  are  not  benign  devices 
through  which  instructional  content  passes  but  devices  that  liberate  learners  -  that 
allow  them  to  experience  and  shape  the  world  around  them  in  new  ways,  perform 
unique  tasks,  and  possibly  a  quire  unique  abilities  previously  unattainable  or,  at 
least,  acquire  generic  skills  more  easily  or  in  other  ways.  This  perspective  has 
important  implications  for  an  educational  system  that  largely  ignores  the  possibility 
that  beneficial  side-effects  accrue  from  learning  activities  (Perkins  and  Salomon, 
1987). 
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Hypotheses 

In  order  to  determine  whether  an  intensive  introductory  course  in  filmmaking 
cultivates  spatial  visualization  and  abstract  reasoning,  and  whether  success  in 
filmmaking  is  directly  related  to  one's  level  of  spatial  visualization  and  abstract 
reasoning  the  following  research  hypotheses  were  formulated: 

1)  Learners  who  receive  intensive  training  and  practice  in  filmmaking  will 
show  greater  improvement,  over  time,  in  their  ability  to  perform  tasks  of 
spatial  visualization  and  abstract  reasoning  compared  with  learners  who  do 
not  receive  training  in  filmmaking. 

2)  A  subject's  spatial  visualization  and  abstract  reasoning  ability  is  directly 
related  to  the  quality  of  film  produced. 

3)  A  subject's  increase  in  spatial  visualization  ability  is  directly  related  to  time 
spent  editing  and  time  spent  engaged  in  cinematography. 

4)  A  subject's  increase  in  abstract  reasoning  ability  is  directly  related  to  time 
spent  editing. 

Method 

In  order  to  test  the  research  hypotheses,  a  quasi-experimental,  nonequivalent 
control  group  design  was  employed.  Subjects  were  conveniently  selected  from 
among  first  and  second  year  university  students  (the  equivalent  of  sophomores  and 
juniors  at  an  American  university)  at  the  University  of  Windsor  in  Windsor, 
Ontario.  An  intensive  introductory  8  month  course  in  16mm  film  production  served 
as  the  treatment  while  a  comparison  group  received  no  film  or  media  instruction. 
Subjects  were  administered  a  series  of  two  tests  four  times  during  the  8  month 
period.  The  tests  were  given  approximately  every  8  weeks.  The  tests  measured 
subjects'  spatial  visualization  and  abstract  reasoning  ability.  The  two  measures 
were  taken  from  the  Differential  Aptitude  Tests  (1980).  The  groups  were  compared 
with  respect  to  their  performance  on  the  tests  as  measured  over  the  8  month  period. 
A  detailed  account  of  the  amount  of  time  spent  by  subjects  in  the  treatment  group  on 
various  filmmaking  activities  was  kept  and  an  assessment  of  the  quality  of  the 
student  films  produced  was  also  made. 

Results 

There  appeared  to  be  compelling  evidence  to  suggest  that  the  introductory 
filmmaking  course  had  no  effect  upon  the  cultivation  of  spatial  visualization  or 
abstract  reasoning.  First,  it  was  the  comparison  group  and  not  the  treatment  group 
that  increased  the  most  with  respect  to  level  of  spatial  visualization.  Second,  there 
was  no  significant  difference  between  the  groups  with  respect  to  their  increase  in 
abstract  reasoning. 

These  apparently  unequivocal  results,  however,  are  muddied  by  a  number  of 
factors.  To  begin  with,  if  the  treatment  were  ineffectual,  one  would  expect  no 
difference  between  the  groups  with  respect  to  the  degree  of  increase  in  spatial 
visualization.  A  learning  effect  is  one  plausible  explanation  for  the  rapid  increase  in 
the  comparison  groups'  level  of  spatial  visualization.  Subjects  were  increasingly 
able  to  remember  the  answers  to  those  question  they  found  easy  and  spend  more 
time  working  on  those  items  of  higher  difficulty  Given  the  nature  of  the  course 
material  and  the  other  courses  comparison  group  subjects  were  taking,  it  is  unlikely 
that  this  increase  can  be  attributed  to  anything  other  than  repeated  exposure  to  the 
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same  instrument.  In  addition,  interview  data  suggested  that  subjects  felt  they  were 
increasingly  better  able  to  "psych  out"  tho  test 

A  partial  ceiling  effect  is  a  plausible  explanation  for  the  comparison  groups  more 
rapid  improvement  relative  the  treatment  group.  The  treatment  groups  average 
score  on  the  first  trial  was  43,1  out  of  a  possible  60.  The  comparison  group  scored 
an  average  of  39.  By  the  second  trial  the  treatment  group  scored  47.8  and  the 
comparison  group  scored  45.1.  By  the  third  trial,  the  treatment  group  scored  48.2 
and  the  comparison  group  46.7. 

It  could  be  argued  that  by  the  time  of  the  second  trial,  a  significant  portion  of  the 
treatment  group  scored  sufficiently  well  that  there  was  little  room  for  improvement 
on  the  subsequent  two  trials.  The  comparison  group,  on  the  other  hand,  scored 
below  the  treatment  group  on  t)   first  two  trials  and  thus  had  more  room  for 
improvement.  As  the  results  show,  the  comparison  group  actually  caught  up  to  the 
treatment  group  by  the  fourth  trial  scoring  49.1  compared  with  49.6  for  the 
treatment  group. 

Unlike  level  of  spatial  visualization,  there  were  no  significant  differences  between 
the  groups  with  respect  to  increases  in  level  of  abstract  reasoning.  As  with  spatial 
visualization,  however,  the  increases  in  abstract  reasoning  may  be  attributed  to  a 
learning  effect  from  the  instrument.  Both  groups  increased  by  nearly  identical 
amounts:  36.9  to  40.0  for  the  treatment  group  and  37.2  to  40.7  for  the  comparison 
group.  There  is  no  evidence  of  a  ceiling  effect  with  the  abstract  reasoning  measure. 

These  results  contradict  those  of  the  Tidhar  and  Sutton-Smith  studies.  It  could  be 
that  age  is  a  salient  variable  in  the  development  of  both  spatial  visualization  and 
abstract  reasoning.  Both  the  Tidhar  and  Sutton-Smith  studies  examined  the 
cognitive  impact  of  filmmaking  on  subjects  aged  6  to  16.  In  both  cases  subjects 
who  received  training  in  filmmaking  displayed  significant  improvement  in  their 
performance  on  spatial  and  abstract  reasoning  tests.  The  current  study,  however, 
deals  with  college-aged  subjects.  Perhaps  the  younger  subjects  of  the  Tidhar  and 
Sutton-Smith  studies  were  at  an  age  when  their  cognitive  skills  were  developing 
rapidly  and  may  have  been  susceptible  to  interventions  that  might  have  quickened 
this  cognitive  development. 

There  is  evidence  to  suggest  that  skill  of  mental  rotation  (an  important  element  in 
spatial  visualization)  begins  to  develop  rapidly  in  children  around  the  age  of  8. 
Prior  to  this  age,  children  make  frequent  mistakes  in  their  attempts  to  predict  the 
appearance  of  rotated  objects  (Inhelder  &  Piaget,  1958).  By  the  time  children  reach 
the  age  of  10,  they  are  able  to  mentally  rotate  objects  nearly  as  well  as  adults  (Kail, 
Pelligrino,  and  Carter,  1980)  although  the  rate  of  mental  rotation  approximately 
doubles  between  this  age  and  adulthood  (Corbollis,  1983). 

If  the  above  is  true,  possibly  the  college-aged  subjects  of  the  present  study,  as  a 
group,  already  possessed  well  developed  and  stable  spatial  visualizations  and 
abstract  reasoning  skills  -  skills  not  likely  to  be  affected  by  any  instructional 
intervention.  The  increases  that  the  filmmaking  and  non-filmrnaking  subjects  did 
experience  were  possibly  due  to  their  increasing  familiarity  with  the  test  and  not 
effects  of  filmmaking. 

It  is  possible,  however,  that  the  treatment  group  was  able  to  perform  the  spatial 
visualization  and  abstract  reasoning  tasks  at  an  increasingly  faster  rate  compared  to 
the  comparison  group.  This  follows  from  Corbollis'  (1983)  assertion  that  change 
in  spatial  visualization  skill  for  teenagers  and  adults  is  reflected  solely  in  terms  of 
speed  of  execution.  Unfortunately,  time  on  task  data  was  not  collected  during  the 
testing  sessions. 
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The  other  focus  of  the  study  was  to  determine  the  extent  to  which  the  cognitive 
skills  of  spatial  visualization  and  abstract  reasoning  were  an  integral  part  of  the 
filmmaking  process.  Certainly  the  results  appear  unequivocal.  Neither  spatial 
visualization  or  abstract  reasoning  was  positively  associated  with  course  grade  or 
final  film  grade.  In  addition,  time  spent  editing  and/or  filming  was  not  related  to 
increases  in  spatial  visualization  or  abstract  reasoning. 

It  may  be  that  with  filmmakers  of  this  age  other  factors  such  as  effort  (time  spent 
making  films  was  directly  related  to  course  grade)  and  conceptual  preparation  (the 
preparation  of  a  clear  and  creative  shooting  script)  are  as  or  more  important  in 
determining  whether  a  high  quality  student  film  is  produced  but  this  can  not  be 
established  given  the  data  at  hand, 

Another  plausible  explanation  for  the  low  correlation  between  level  of  spatial 
visualization  and  abstract  reasoning  and  a  subject's  final  grade  should  be  noted.  A 
close  examination  of  the  final  grade  reports  showed  that  non  filmmaking  activities, 
such  as  essays,  scripts,  and  exams  accounted  for  more  of  the  final  grade  than 
filmmaking  assignments.  In  other  words,  the  final  course  grade  was  not  a  good 
measure  of  filmmaking  ability, 

Implications  and  Suggestions  For  Future  Research 

The  results  of  this  study  should  not  deter  future  research  into  the  effects  of  active 
expression  through  visual  media.  Nor  should  these  results  call  into  question  the 
inclusion  of  filmmaking  in  high  school  and  university  curricula;  however,  further 
investigations  of  filmmaking  and  its  cultivation  of  spatial  and  abstract  reasoning 
skills  in  adults  may  not  to  be  fruitful  However,  additional  work  is  necessary  to 
better  understand  the  relationship  between  these  cognitive  skills  a^  1  filmmaking, 
Future  research  should  also  strive  to  identify  more  meaningful  ways  in  which  the 
activity  of  filmmaking  changes  the  filmmaker.  With  this  information  in  hand, 
filmmaking  courses  can  be  more  wisely  incorporated  into  school  curricula, 

Clearly,  there  is  need  for  conceptual  research  to  help  articulate  meaningful 
dimensions  of  impact.  Such  research  would  form  the  basis  for  further  empirical 
work.  Within  the  cognitive  domain,  what  other  skills  might  likely  be  enhanced  by 
the  manipulation  of  visual  and  aural  information.  Perhaps  filmmaking  effects  the 
way  visual  data  are  stored  and  retrieved. 

These  questions  have  greater  importance  now  that  small  format  video  technology  is 
so  readily  accessible.  Millions  of  people  experience  significant  parts  of  their  lives  - 
their  vacations,  their  child's  first  birthday,  their  daughters  wedding  -  by  peering 
through  the  viewfinder  of  a  video  camera.  They  then  "relive"  these  experiences  by 
repeated  showings  of  the  images  and  sounds  they  have  collected.  How  are  one's 
perceptions  of  these  experiences  altered  by  the  technology  used  to  collect  and  view 
them? 

It  is  also  important  to  explore  the  effect  of  filmmaking  in  ways  other  than  cognitive 
skill  development  or  in  terms  of  how  it  shapes  and  alters  our  perceptions.  For 
example,  how  might  filmmaking  develop  team  building  skills  or  facilitation  skills. 
How  might  filmmaking  encourage  dialogue  within  a  community?  During  the  late 
1960's  and  early  1970's,  the  National  Film  Board  of  Canada  embarked  on  an 
ambitious  plan  to  teach  filmmaking  to  Canadian  native  groups.  The  natives  used 
their  new  skills  to  produce  films  about  their  goals  and  aspirations  and  the  many 
problems  they  faced.  The  project,  in  essence,  acted  as  a  needs  assessment  exercise 


for  the  natives.  It  facilitated  discussion  among  the  native  community  and  helped  the 
community  to  identify  problems  and  possible  solutions. 


Another  related  and  important  question  is  the  role  media  production  should  play  in 
a  media  literacy  curriculum.  The  province  of  Ontario,  for  example,  has  recently 
completed  a  major  high  school  curriculum  initiative  in  the  area  of  media  literacy 
(Ontario  Ministry  of  Education,  1989).  The  report  suggests  that  "hands-on"  media 
experience  may  facilitate  the  development  of  students*  overall  understanding  of  how 
the  media  affect  their  lives.   However,  more  work  needs  to  be  done  to  confirm  this 
and  determine  the  kinds  of  media  production  experiences  best  suited  to  this  kind  of 
curriculum  objective. 

In  summary,  despite  the  non-significant  findings  of  this  study  it  is  important  that 
we  realize  the  value  of  media  not  solely  in  terms  of  delivering  instructional  content. 
We  need  to  see  media  as  instruments  that  allow  learners  to  engage  in  unique  and 
educative  activities  and  that  allow  learners  to  experience  the  world  around  them  in 
new  and  exciting  ways. 
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Abstract 

Although  both  learner  control  and  feedback  have  been  heavily 
researched,  very  little  research  has  been  conducted  on  giving 
learners  control  over  the  feedback  which  they  receive.  The 
purpose  of  this  study  was  to  examine  the  effect  of  learner 
control  over  feedback  in  a  CAI  lesson.  Subjects  used  one  of 
four  CAI  programs  which  provided  either  program  control  or 
learner  control  over  verification  or  elaboration  feedback. 
Results  indicated  that  subjects  who  received  elaboration 
feedback  during  instruction  performed  better  than  students  who 
received  verification  feedback.  Type  of  control  did  not  have 
a  significant  influence  on  performance.  Implications  for  the 
design  of  CAI  are  discussed. 

Introduction 

While  a  great  deal  of  research  has  been  conducted  on 
learner  control  and  on  feedback,  few  studies  have  been 
conducted  to  determine  if  learner  control  of  feedback  will 
have  an  effect  on  student  performance  and  attitude.  Computers 
now  make  it  possible  to  allow  learners  to  control  the  amount 
of  feedback  in  instruction.  But  it  is  not  clear  whether 
giving  learners  control  of  feedback  is  beneficial.  Some 
writers  have  suggested  that  the  "mere  illusion  of  control" 
significantly  improves  motivation  and  performance  (Perlmuter 
and  Monty,  1977) ,  while  others  have  concluded  "there  is  little 
support  from  the  research  literature  that  offering  students 
control  will  lead  to  increased  learning"  (Carrier,  1984,  p. 
17)  . 

Several  researchers  report  advantages  for  allowing 
learners  to  have  control  in  computer-assisted  instruction. 
Learner  control  over  the  instructional  strategy  of  a  CAI 
program  has  positively  influenced  retention  of  information  and 
student  interest  (Newkirk,  1973).  Learner  control  over  review 
options  (Kinzie,  Sullivan,  &  Berdel,  1989)  and  contextual 
properties    (Ross    &    Morrison,     1989)     in    CAI    lessons  has 
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significantly  increased  test  performance.  Hansen  (1974)  found 
that  learner  control  over  feedback  in  a  CAI  lesson  decreased 
student  anxiety  about  learning,  while  others  have  reported 
that  both  feedback  and  learner  control  in  CAI  increased 
student  performance  and  attitude  (Schloss,  Wisniewski,  & 
Cartwright,  1988;  Steinberg,  Baskin,  &  Hofer,  1986). 

According  to  Clariana,  Ross,  and  Morrison  (1991)  feedback 
is  an  important  variable  that  is  often  ignored  in  CAI. 
Researchers  have  reported  that  feedback  increases  learner 
performance  and  reduces  program  errors  (Anderson,  Kulhavy  & 
Andre,  1972;  Kulhavy,  1977;  Kulhavy,  Yekovich,  &  Dyer,  1979). 

Feedback  is  a  unit  of  information  with  two  components, 
verification  and  elaboration  (Kulhavy  &  Stock,  19' 3)  . 
Verification  is  the  simple  dichotomous  judgment  that  an 
initial  response  was  right  or  wrong.  Elaboration  consists  of 
all  substantive  information  contained  in  the  feedback  message. 
Collins,  Carnine  &  Gersten  (1987)  demonstrated  that  when 
verification  and  elaboration  were  given  in  CAI,  performance 
was  significantly  higher  for  elaboration,  while  the  time  to 
complete  was  similar.  In  addition,  a  meta-analysis  conducted 
by  Bangert-Drowns,  Kulik,  Kulik,  and  Morgan  (1991)  indicated 
that  elaboration  feedback  produced  greater  effects  for 
learning  than  verification  feedback. 

The  purpose  of  this  study  was  to  examine  the  effect  of 
learner  control  over  feedback  in  computer-assisted 
instruction.  The  independent  variables  were  type  of  control 
(learner  or  program)  and  level  of  feedback  (verification  or 
elaboration) .  The  dependent  variables  were  performance  on  a 
posttest,  attitude  toward  the  program,  and  time  to  study 
feedback. 

Method 

Subjects 

Subjects  were  100  undergraduate  education  majors  enrolled 
in  an  educational  psychology  class  at  a  large  southwestern 
university.  Data  for  93  out  of  the  original  100  subjects  were 
included  in  the  analyses  of  the  results  because  scores  on  one 
or  more  measures  were  unavailable  for  the  remaining  seven 
subjects. 

Materials 

Materials  used  in  this  study  were  four  CAI  lessons,  a 
posttest,  and  an  attitude  questionnaire.  The  CAI  lessons  were 
developed  using  a  software  package  called  the  Presenter 
(Behrens    &    Stock,    1990) .      All    lessons   provided   the  same 


9 

ERIC 


eo:'  638 


Control  of  Feedback 


3 

information,  e>amples,  and  practice  on  the  concepts  of 
reliability  and  validity.  The  lessons  were  based  on  the  text 
Topics  ia  Measurement;  Reliability  and  Validity  by  Dick  & 
Hagerty  (1971).  Information  and  examples  were  presented  in 
sections  of  five;  screen?*  of  text,  followed  by  eight,  five- 
al cernative,  multiple-choice  questions.  This  cycle  continued 
lor  "i.  total  of  25  screens  of  text  and  40  questions. 

The  differences  in  tTn»  CAI  lessons  were  based  on  type  of 
control  (program  -md  Jaarner)  and  level  of  feedback 
(verification  ~nd  elaboration) .  Under  program  control,  the 
ooinpu-e;-  vvroyraa  administered  one  of  the  two  feedback 
conditions  * u>  .:j.ai',i ..ally .  Under  learner  control,  subjects 
decided  if  they  tmitfed  to  receive  feedback.  Under  the 
condition  of  verification,  a  learner  was  told  only  if  a 
response  was  cor :e.:t  or  incorrect;  while  under  elaboration,  a 
learner  was  told*  if  a  response  was  correct  or  not,  the  correct 
answer,  and  a  &nert  explanation. 

Subjects  using  the  program  control/verification  lesson 
were  always  provided  with  the  feedback  message  "yes,  you  are 
correct"  or  "no,  you  are  incorrect"  after  each  practice 
question.  Subjects  using  the  learner  control/verification 
lesson  were  asked  "would  you  like  to  check  your  answer?"  after 
each  question.  If  the  response  was,  "yes",  then  the 
appropriate  verification  feedback  was  presented.  If  the 
response  was,  "no",  the  program  continued  with  the  next 
question  or  screen  of  text. 

Subjects  using  the  program  control/elaboration  lesson 
always  received  verification  information,  followed  by  the 
correct  answer  and  a  short  explanation  after  each  practice 
question.  Subjects  using  the  learner  control/elaboration 
lesson  were  asked  "would  you  like  to  check  your  answer?"  after 
each  question.  If  the  response  was,  "yes",  verification 
appeared  as  described  above.  Th^  lesson  then  asked,  "would 
you  like  an  explanation?"  If  the  response  was,  "yes",  the 
correct  answer  and  an  explanation  appeared  before  the  program 
continued.  If  the  response  was  "no",  the  lesson  continued 
with  the  next  question  or  screen  of  text. 

In  addition  to  the  four  CAI  lessons,  a  posttest  and  an 
attitude  questionnaire  were  developed.  The  posttest  consisted 
of  the  same  40  questions  previously  given  as  practice  but 
presented  in  a  random  order.  The  reliability  of  the  posttest 
was  calculated  at  .69  using  the  Kuder-Richardson  20  formula. 
The  attitude  questionnaire  consisted  of  ten  items  measuring 
student  satisfaction,  enjoyment,  perception  of  control,  and 
feeling  toward  feedback.  The  questionnaire  used  a  five  point 
Likert-type  scale.     Both  measures  were  administered  on  the 
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computer.  In  addition,  the  computer  automatically  recorded 
the  number  of  seconds  each  subject  spent  studying  feedback 
messages. 

Procedure 

Before  subjects  arrived  to  participate  in  the  study,  an 
experimenter  prepared  the  computer  laboratory  by  installing 
one  of  the  four  lessons  into  each  computer.  Upon  arrival  to 
the  computer  room,  each  subject  was  randomly  assigned  to  one 
of  the  four  experimental  conditions.  All  four  of  the 
conditions  were  present  at  each  experimental  session. 

The  experimenter  gave  a  short  introduction  on  general 
procedures  and  told  subjects  that  instructions  were  included 
in  the  program.  They  were  not  told  that  the  programs  were 
different.  Subjects  were  told  that  the  lesson  was  on 
reliability  and  validity  and  stressed  the  importance  of  the 
material  for  them  as  future  teachers.  Subjects  were  also  told 
that  they  would  have  to  pass  a  test  at  the  end  of  the  lesson 
in  order  to  receive  points  toward  their  final  course  grade. 
Subjects  then  proceeded  with  their  individual  lessons.  Upon 
completion  of  the  lesson,  each  subject  completed  the  attitude 
questionnaire  and  the  posttest  on  the  computer.  They  were 
given  as  much  time  as  they  needed  to  complete  the  lessons  and 
the  criterion  measures.  Most  subjects  completed  the  study 
within  a  50-minute  class  period. 

Design  and  Data  Analysis 

The  design  was  a  2  x  2  factorial  with  type  of  control 
(learner  or  program)  and  level  of  feedback  (verification  or 
elaboration)  as  the  independent  variables.  The  dependent 
variables  were  performance,  attitude,  and  feedback  study  time. 

Analysis  of  variance  (ANOVA)  was  used  to  test  for 
differences  between  groups  on  performance  and  feedback  study 
time.  A  multiple  analysis  of  variance  (MANOVA)  was  used  to 
test  for  differences  between  groups  on  the  attitude 
questionnaire.  The  MANOVA  was  followed  by  univariate  analysis 
for  each  question.  An  alpha  level  of  .05  was  set  for  all 
statistical  tests.  In  addition,  effect  sizes  (ES)  were 
calculated. 

Results 

Performance 

Mean  scores  and  standard  deviations  for  performance  can 
be  found  in  Table  1.     These  data  indicate  that  the  mean  for 
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subjects  who  received  verification  feedback  was  2b. 11,  while 
the  mean  for  subjects  who  received  elaboration  feedback  was 
31.15.  The  mean  score  for  the  program  control  group  was  28. 5S 
and  the  mean  score  for  the  learner  control  group  was  27.67. 

Analysis  of  the  posttest  data  indicated  that  level  of 
feedback  had  a  significant  effect  <n  performance  £(1,  89)  = 
39.47,  p_  <  .05,  M3e  =  21.41,  ES  *  1.09.  Subjects  who  received 
elaboration  feedback  performed  better  than  those  receiving 
verification  feedback,  regardless  of  the  type  of  control 
provided.  The  difference  for  type  of  control  and  the  feedback 
by  control  interaction  were  not  significant. 

Table  1 

Means  and  Standard  Deviations  for  Performance 


Type  of  Feedback 


Control 

Verification 

Elaboration 

Totals 

Program 

M 

25.17 

32.00 

28.59 

SD 

4.72 

3.66 

4.19 

Learner 

M 

25.  04 

30.29 

27.67 

SD 

4.79 

5. 18 

4.98 

Totals 

M 

25.11 

31.15 

28.15 

SD 

4.76 

4.42 

4.63 

Highest  possible  score  was  40.  Cell  sizes  were  24  for  learner 
Control/elaboration  and  23  for  the  other  three  conditions. 


Attitude 

Analysis  of  the  10-item  attitude  questionnaire  data 
revealed  a  significant  MANOVA  effect  for  level  of  feedback, 
F(10,  80)  =  4.93,  p_  <  .05.  Follow-up  univariate  analyses 
indicated  a  significant  difference  between  feedback  conditions 
for  item  10  (I  would  have  liked  to  have  more  feedback  about  my 
answers),  F(l,  89)  =  39.48,  p_  <  .05,  Mj3e  =  1.04,  ES  =  .55. 
Subjects  who  received  verification  (M  =  1.35,  SJ)  =  0.71) 
indicated  a  greater  desire  to  receive  more  feedback  than  those 
who  received  elaboration  (M  =  2.66,  SD  ■  1.22)  .  No  other 
significant  differences  were  found  on  the  attitude 
questionnaire. 

Feedback  Study  Time 

The  means  and  standard  deviations  for  feedback  study  time 
were  calculated  in  seconds.  The  largest  differences  in 
feedback  study  time  were  between  subjects  who  received 
verification  (M  =  70.96,  S_fi  =  20.56)  and  those  who  received 
elaboration  (M  =  287.67,  SD  =  124.61)  .    The  mean  for  *urjects 
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who  received  program  control  was  155.07  (&J>  ■  152.35)  and  the 
mean  for  those  who  received  learner  control  was  195 •  76  (SD  - 
130.23)  . 

Analysis  of  the  data  for  feedback  study  time  revealed  a 
signxficant  effect  for  level  of  feedback,  £(1,  89)  =  132.60, 
E  <♦  05,  MSe  -  8225. 63,  ES  =  .75.  Subject**  who  received 
elaboration  spent  an  average  of  217  seconds  mora  when  studying 
feedback  than  subjects  who  received  verification.  Type  of 
control  did  not  significantly  affect  feedback  study  time. 

Dis<  ission 

The  purpose  of  this  study  was  to  investigate  the  effects 
of  learner  control  over  feedback  in  a^  instructional  computer 
program.  Results  suggest  that  students  who  receive 
elaboration  feedback  during  instruction  will  perform  better 
than  students  who  receive  verification  feedback.  This  is 
consistent  with  other  research  which  indicates  that 
elaboration,  rather  than  verification,  produces  greater 
effects  in  learning  (Bangart-Drowns,  Kulik,  Kulik,  &  Morgan, 
1991;  Collins,  Gamine  &  Gersten,  1987). 

There  are  several  possible  reasons  why  elaboration 
affected  performance  in  the  current  study.  One  likely  reason 
is  that  performance  was  improved  due  to  the  increased  amount 
of  information  in  the  elaborated  feedback  message.  Students 
who  received  elaboration  were  provided  with  the  correct  answer 
and  an  explanation  of  that  answer  after  each  practice 
question.  However,  subjects  in  the  verification  condition 
were  only  told  whether  their  answers  were  correct  or 
incorrect.  It  is  likely  that  students  who  received 
elaboration  used  this  additional  information  to  correct  errors 
made  on  practice  items. 

Another  factor  that  may  have  contributed  to  the  positive 
ef  'ect  for  elaboration  is  the  increased  in  time  spent  in 
-wUdying  feedback.  Students  who  received  elaboration  spent 
more  time  studying  feedback  than  those  who  received 
verification.  Additionally,      students      who  received 

verification  indicated  a  desire  to  have  more  feedback  during 
instruction. 

The  results  of  this  study  suggest  some  implications  for 
the  design  of  CAI.  Instructional  designers  should  consider 
providing  different  feedback  messages  in  CAI  lessons  depending 
on  student  responses.  Verification  feedback  could  be  provided 
to  students  when  their  initial  response  to  an  itejn  is  correct, 
but  it  seems  vital  to  provide  elaborated  feedback  when  an 
initial  response  if  incorrect.      CAI  has  the  capability  to 
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provide  differing  feedback  messages  depending  upon  student 
responses. 

The  current  study  also  has  some  implications  for  future 
research  on  feedback  and  learner  control.  Future  studies 
should  investigate  whether  different  feedback  messages  will 
increase  performance  when  the  feedback  is  based  on  student 
responses.  Research  on  the  effect  of  verification  and 
elaboration  feedback  should  be  conducted  on  differing  learning 
outcomes.  Furthermore,  future  research  should  examine  if 
learner  control  based  on  student  responses  during  practice 
will  affect  performance.  Implementation  of  these  suggestions 
will  assist  us  in  determining  the  conditions  under  which 
different  types  of  feedback  and  learner  control  provide  the 
greatest  benefits. 
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Abstract 


This  paper  proposes  strategies  of  motivation  in  CBI  derived  from  motivational  foundations 
of  cognitive,  affective  and  social  learning  theories.  The  assumption  made  here  is  that  motivation 
and  achievement  interact  in  ways  that  warrant  a  "holistic"  examination  of  strategies  incorporated 
into  CBI.  Thus  motivation  in  CBI  is  not  solely  a  function  of  affect-  it  can  be  strengthened  by 
cognitive  and  social  motivational  influences.  In  this  holistic  motivational  framework,  CBI  must  serve 
to  improve  learners'  expectations  of  success,  enhance  feelings  of  self-efficacy,  increase 
perception  of  control  and  help  learners  make  positive  effort,  ability  and  strategy  attributions.  The 
role  of  theories  and  models  of  motivation  and  their  relevance  to  CBI  is  explored,  and  ways  in  which 
computers  are  amenable  to  the  incorporation  of  strategies  of  motivation  are  explicated. 
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Motivational  atrataglat  In 
computar-baaad  Instruction: 
Soma  laaaona  from  theories  and 
modala  of  motivation 


Why  do  children  enjoy  computers? 
Why  do  they  choose  to  return  frequently 
and  voluntarily  to  computer-based  tasks? 
To  persist  with  problem-solving  or  drills  undl 
they  have  achieved  mastery?  To 
demonst/ate  task  engagement  while 
working  on  the  computer?  Task 
persistence,  perceived  control,  level  ol  task 
engagement,  cognitive  effort  expended, 
attitude  towards  learning,  cognitive 
monitoring,  expectancies  of  success,  self- 
efficacy  and  self-worth  have  been 
traditionally  studied  under  a  component  of 
learning  labelled  variously  as  achievement 
motivation,  continuing  intrinsic  motivation  to 
learn,  intrinsic  motivation,  need  for 
achievement.  In  the  classroom  learning 
context,  extensive  theoretical  and  empirical 
work  has  revealed  the  complex  nature  of 
factors  affecting  intrinsic  and  extrinsic 
motivation  to  learn  (Brophy,  1983).  This 
research  has  contributed  towards  a  variety 
of  interventions  with  demonstrated 
empirical  merit  in  improving  achievement 
and  motivation. 

The  intent  of  this  paper  is  to  identify 
motivational  strategies  pertinent  to  a  holistic 
approach  to  motivation  in  computer-based 
instruction.  Relevant  theories  which 
encompass  social,  cognitive  and  affective 
foundations  of  motivation  to  team  are 
summa.lzed,  and  implications  arising  from 
such  theories  in  strengthening  motivational 
aspects  of  CBI  are  explicated.  An  attempt  Is 
made  to  answer  the  folfcwinn  ^astions: 
What  is  the  relationship  between  theories 
of  motivation  and  computer-based 
motivational  strategies?  How  can  the 
unique  attributes  of  the  computer  be  used 
to  create  software  incorporating  empirically- 
based  findings  and  guidelnes  from  the 
theories  of  motivation?  Finally,  what 
functions  must  motivational  aspects  of  CBI 
serve? 

Computers  and  Motivation 

At  a  time  when  there  is  nationwide 
concern  about  declining  standardized 
scores,  educators  are  advocating  intelligent 


use  of  technology  to  make  learning 
efficient,  relevant,  contemporary  and 
interesting  (Clai*  &  Salomon,  1986;  Winn, 
1989).  Motivational  factors  are  an  important 
mediator  of  performance,  and  are 
considered  Instrumental  in  the  expenditure 
of  cognitive  effort  (Bandura,  1977;  Co  mo 
and  Mandinach,  1983).  What  students 
perceive,  encode  and  assimilate  in 
classrooms,  in  social,  affective  and 
academic  realms  is  a  function  of  their 
motivational  states  (Thomas,  1980).  These 
conclusions  point  towards  the  power  of  the 
environment  in  influencing  motivation. 
Although  learning  from  CBI  is  not  the 
panacea  to  the  complex  issue  of  academic 
achievement,  It  can  provide  environments 
which  are  non-threatening,  enjoyable  and 
cogritively  stimulating,  while  boosting 
achievement. 

The  unique  attributes  of  computers 
which  make  them  amenable  to  motivating 
instruction  are  specially  apparent  when 
compared  to  the  manner  in  which 
classrooms  are  structured.  Classrooms  are 
characterized  by  a  diversity  of  cognitive  and 
affective  aptitudes  (Cronbach  and  Snow, 
1977),  In  me  absence  of  adequate  time 
and  attention  by  a  single  teacher  managing 
an  entire  class,  students  are  left  on  their 
own  to  manage  the  multitude  of  stimuli 
generated  in  the  classroom.  Learning  in 
classrooms  is  social  and  open  to  evaluation 
by  others  (Ames  &  Ames,  1984).  Without 
appropriate  intervention,  students  may 
develop  stable  negative  attributions  'jhich 
impinge  on  performance.  Teacher 
expectations  of  students  also  serve  as  a 
sen-fulfilling  prophecies  in  student 
performance  and  behavior,  affecting 
students'  self-concept,  effort  expenditure 
and  hence  achievement  motivation 
(Brophy,  1985;  Ecles  and  Midgeley,  1989). 

Well-designed  computer-based 
instruction  offers  an  environment  which  can 
offset  some  of  these  negative  influences. 
By  virtue  of  representing  modern 
technological  media,  computers  have  an 
appeal  that  evokes  curiosity  (Brown,  1986). 
Students  approach  computers  with  a 
positive  initial  attitude,  making  them 
effective  attention  gaining  devices  (to'  eg., 
Hativa,  1988).  In  a  meta-analysis  of  51  CAI 
studies,  Kulk,  Bangert  and  WiWams  (1983) 
found  that  CAI  improved  attitudes  toward 
coursework  and  the  computer;  the  authors 
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sunrise  that  the  decreased  time  in 
completing  CAI  assignments  may  be 
attributed  to  motivational  influences  of  the 
computer.  When  asked  to  Identify  and  rate 
what  they  liked  about  CBI  and  classroom 
instruction,  sixth-grade  students  rated 
motivational  attributes  of  computers  higher 
than  classrooms  (Perez  &  White,  1985).  In 
the  same  study,  students  identified  content 
and  computer  related  (motivational)  aspects 
of  CBI,  whereas  only  content  related 
features  were  identified  for  classroom 
instruction.  Weisz  and  Cameron  (1989) 
provide  evidence  that  computers  are  liked 
because  of  the  control  they  offer. 
Robertson,  Ladewig,  Strickland  and 
Boschung  (1987)  found  that  using 
supplementary  microcomputer  instruction 
yielded  significantly  higher  global  self- 
esteem  ratings,  leading  the  authors  to 
conclude  that  Increased  achievement 
which  resulted  from  the  use  of  CAI  mav 
have  contributed  to  this  increment. 

Among  the  most  appealing  attributes 
of  computers  Is  their  abiity  to  individualize 
instruction  and  make  it  interactive;  His 
possible  to  engage  learners  longer  and 
more  meaningfully  with  Individualized 
interaction  (Jonassen,  1984).  The 
cognitive  motivational  effects  of  CBI  can  be 
observed  simultaneously  In  this  adaptive 
framework:  instruction  can  be  so  designed 
that  students  experience  a  feeling  of 
success,  as  reflected  in  their  performance, 
and  develop  positive  expectancies  of 
success.  CBI  gives  students  greater 
opportunities  of  choice  than  a  traditional 
classroom.  Immediate,  consistent  and  non- 
judgemental  feedback,  an  asset  in  learning 
from  a  cognitive  viewpoint  influences 
significant  determinants  of  achievement 
behavior  such  as  self-efficacy  and 
expectancies  of  success  (Hannatin  & 
Dalton,  1984).  Because  computers  are 
viewed  as  being  fairer  than  teachers, 
students  with  a  low  self -Image  in  class  fee! 
less  vulnerable  to  social  comparisons  while 
working  with  computers.  The  benefits  of 
immediate  feedback  in  CBI,  although 
seemingly  obvious  by  now,  should  not  be 
underestimated:  it  enhances  perception  of 
control,  and  maintains  arousal.  Finally,  it 
allows  students  to  work  at  their  own  pace 
reducing  the  anxiety  resulting  from 
speeded  learning  and  tests. 


Investigations  on  computer-based 
motivational  strategies  (eg.,Malone,  1981) 
have  attempted  to  Isolate  variables  which 
contribute  to  making  learning  interesting 
and  fun.  The  focus  of  this  Hne  of  research 
has  been  an  In-depth  analysis  of  computer 
games  to  unravel  the  mystery  of  sustained 
interest  in  learning  from  computers, 
resulting  in  a  conceptual  framework  of 
motivation  in  computer-based  instruction. 
However,  this  resewch  has  also  culminated 
in  questions  regarding  the  validity  of 
computer-based  Instructional  games  in  all 
learning  situations  (Lepper  &  Chabay, 
1985):  should  all  Instructional  software  be 
designed  in  the  form  of  games  to  make  CBI 
motivationally  effective?  Is  interest  or 
intrinsic  motivation  observed  during  games 
sustained  over  time?  Is  performance 
specific  to  the  fantasy  that  students  learned 
under?  Further,  Lepper  and  Hodell  (1989) 
claim  that  soma  instructional  games  have 
more  negative  than  positive  features,  and 
may  provide  Inordinately  high  levels  of 
excitement,  which  are  detrimental  to 
learning. 

What  then,  are  the  determinants  of 
motivation  in  tasks  other  than  computer- 
based  games?  In  the  ensuing  discussion, 
some  such  determinants  of  motivation  are 
hypothesized  based  on  empirical  and 
theoretical  support. 

The  cognitive,  affective  and  social 
foundations  of  motivation 

Two  assumptions  form  the  bases  for  a 
holistic  approach  to  motivation  in  CBI.  First, 
achievement  and  motivation  are  fostered 
through  reciprocally  interactive  processes, 
information  is  attended  to,  selected, 
transformed  and  assimilated  with  the 
learners*  existing  schema,  causing  shifts  in 
the  knowledge  structures  of  the  learner; 
this  process,  In  turn  affects  values,  beliefs, 
expectations  for  future  tasks,  and  hence 
selective  perception  of  Incoming  stimuli. 
Sucn  interactions  illuminate  the  significance 
of  the  environment-  the  structure  of 
knowledge  existing  outside  the  learner. 
Salomon  (1979)  Illustrates  this  elegantly  in 
his  theory  of  "reciprocal  Interactionism"  in 
values,  cognitions,  expectations  and 
beliefs  between  tho  medium  and  the 
Lamer. 

Second,  motivation  is  differentially 
influenced  by  cognitive,  social  and  affective 
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factors  among  learners  with  diverse  learner 
characteristics.  Several  theories  of 
motivation  have  examined  affective  sources 
of  motivation;  while  valuable  Insight  Is 
provided  by  the  role  of  Individual  affective 
variables  (eg.,  persistence,  locus  of  control, 
attributionaJ  style),  these  do  not  embody 
the  holistic  nature  of  academic  learning,  in 
which  affective,  cognitive  end  social  forces 
reinforce  eachother  continually. 

CognHlva  foundations  of  motivation:  An 
explicitly  cognitive  approach  to  motivation 
has  been  the  focus  several  theorists 
(Corno,  1986;  Corno  &  Rohmkemper, 
1989;  Corno  &  Mandinach,  1983; 
Zimmerman  &  Pons,  1986).  Corno  & 
Mandinach  (1983)  believe  that  intrinsic 
motivation  is  a  function  of  the  level  of 
cognitive  engagement  in  a  task.  It  is  an 
outcome  of  self-regulated  learning  (SRL), 
which  they  define  as  "an  effort  put  forth  by 
students  to  deepen  and  manipulate 
associative  network  in  content  area,  and  to 
monitor  and  improve  the  deepening 
process"  (p.  95).  In  this  conceptual 
approach,  SRL  not  only  fosters  Increased 
learning,  but  increased  motivation  as  well. 
The  authors  consider  it  the  highest  form  of 
cognitive  engagement,  necessary  for 
continued  intellectual  performance.  In 
Corno  and  Mandinach's  model,  SRL 
processes  consist  of  five  component 
processes:  alertness,  selectivity, 
connecting,  planning  and  monitoring. 
Alertness  and  monitoring  are  largely 
information  acquisition  processes,  whereas 
the  remaining  three  are  transformation 
processes.  Both  these  classes  of 
information  processing  have  a 
metacognitive  component. 

Zimmerman  &  Martjnez-Pons  (1988) 
state  that  self-regulated  learners  are 
"metacognrtivery,  motivattonaHy,  and 
behavioraHy  active  participants  in  their  own 
learning  process."  (p.  284).  They  plan, 
organize,  self-instruct,  and  self-evaluate  at 
various  stages  during  the  acquisition 
process,  while  selecting,  structuring,  and 
even  creating  social  and  physical 
environments. 

Corno  (1986)  has  also  focused  on 
the  metacognitive  control  components  of 
sett-regulated  learning-  cautioning, 
however,  that  metacognitive  components 
are  necessary  but  insufficient  for  self- 


regulated  learning.  From  this  viewpint, 
metacognitive  efforts  to  monitor  and  control 
concentration  and  affect,  in  conjunction 
with  the  appNcation  of  effective  cognitive 
strategies  serve  the  purpose  of  efficiency 
and  effectiveness. 

Social  learning  theory  and  achievement 
motivation:  Social  leamkj  theorists  such  as 
Bandura  (1977;  1982)  and  Schunk  (1984), 
maintain  that  the  underlying  faoiStator  of 
cognitive  effort  expended  on  learning  a 
skid-  the  determinant  of  motivation,  is  solf- 
etflcacy.  Self-efficacy  is  described  as 
personal  Judgements  about  one's  capability 
to  organize  and  implement  actions  in 
specific  situations  that  may  contain  novel, 
unpredictable  and  possibly  stressful 
features."  (Schunk,  1984,  p.  48).  In  this 
theoretical  framework,  self-efficacy  affects 
the  choice  of  activity  and  the  mental  effort 
expended  in  understanding,  monitoring 
and  rehearsing.  Further,  low  self-efficacy 
implies  less  eager  participation  in  an  activity, 
and  vice- versa  (Bandura,  1977),  Factors 
affecting  self-efficacy  appraisal  include 
purpose  of  instruction,  perceived  ability, 
task  difficulty,  effort,  assistance,  task 
outcomes,  perceived  similarity  to  models 
and  persuader  credibility  (Schunk,  1989). 

Another  perspective  on  motivation 
originates  from  the  concept  of  cooperative 
learning,  extensively  researched  for  its 
motivational  and  achievement  benefits.  For 
example,  Johnson  and  Johnson  (1985) 
view  motivation  as  resulting  from  processes 
of  cooperative  learning,  providing  evidence 
that  learners  attribute  group  success  to 
abilty,  which  in  turn  promotes  achievement. 
In  their  research  cooperative  learning  has 
emerged  as  a  strong  predictor  of  academic 
success  when  compared  to  other  forms  of 
social  interdependence  such  as 
competitive  and  cooperative  environments. 

From  a  social  learning  perspective 
then,  improved  achievement  results  in 
greater  self-efficacy  (Schunk,  1989),  which 
is  instrumental  in  expendHure  of  effort, 
providing  an  impetus  to  achievement. 

Affect  as  a  determinant  of  achliwflment 
motivation:  Weiner  (1979;  1989),  proposes 
that  Individuals  seek  causes,  or  attributions 
of  past  success  or  failure,  which  in  turn 
dfctates  their  future  expectancies  of 
success  and  achievement  behavior.  These 
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causal  attributions  fall  into  four  categories: 
ability,  task  difficulty,  effort  and  luck,  and  He 
along  three  causal  dimensions:  intemal- 
extemaJ.  causal  staoilty,  (eg.,  stable- 
unstable),  and  contfoHabilty  (controllable 
vs.  uncontrollable  events).  Attributions  and 
causal  dimensions  Interact  In  afferent  ways; 
for  example,  aptitude,  effort  and  health  are 
internal  factors  (within  the  volitional  control 
of  learner),  whereas  fuck,  task  difficulty, 
help  from  others  fall  within  the  external 
dimension.  Failure  ascribed  to  internal 
factors  (eg.,  lack  of  ability)  produces  tower 
expectancies  of  success.  Similarfy.abiityis 
considered  a  stable  dimension  (versus 
effort),  and  prior  outcomes  ascribed  to 
abllty  are  more  predictive  of  future 
performance  than  ascriptions  to  effort. 
Finally,  whereas  effort  is  under  volitional 
control,  ability  is  an  uncontrollable 
dimension. 

Another  variable  which  has  been 
universally  acknowledged  to  affect 
motivation  is  perception  of  control  (Schunk, 
1984;  Stipek&Weisz,l981;  Thomas 
1980),  Wetez  and  Cameron  (1985)  define  it 
as  the  ability  to  cause  an  intended  event. 
McCombs  (1984)  states  that  even  if  ability 
to  succeed  Is  present,  achievement  and 
motivation  win  be  impaired  without  a 
perception  of  control.  Perceived  control 
increases  judgements  of  self-efficacy  or 
personal  competence,  implying  positive 
effect  on  motivation  and  achievement. 

Thft  rote  of  thft  RpH-svstam  in  intrinsic 
motivation:  Recentfy,  motivation  has  been 
construed  as  a  function  of  cognitive  and 
affective  components,  representing  a 
holistic  approach  to  learning.  McCombs 
(1982;  1984),  has  emphasized  the  role  of 
perceived  self-efficacy  and  personal  control 
in  her  proposed  definition  and  perception 
of  "continuing  motivation  to  learn",  which  is 
defined  as  a  "dynamic,  Internally  mediated 
set  of  metacognttive,  cognitive  and 
affective  processes  (Including 
expectations,  attributions,  beliefs  about  the 
self  and  the  learning  environment)  that  can 
influence  a  student's  tendency  to 
approach,  engage  In  and  expend  effort  in, 
and  persist  In  learning  tasks  on  a 
continuing,  self -directed  basis"  (p.  200).  A 
metacognhtve  self-awareness  Is  an  integral 
component  of  this  model.  McCombs 
recommends  several  interventions  aimed  at 


the  sell  to  optimize  seK-regu'ated  learning. 
These  consist  of  support  to  students  which 
can  focus  on  both  Internal  self-system 
processes  (eg.,  helping  students  change 
dysfunctional  setf-beliefs,  values  or  motives 
via  cognitive  restructuring  approaches),  and 
on  rrtodifications  to  the  external  learning 
environments  (eg.,  helping  students 
acquire  strategies  for  self-awareness,  self- 
evaluation  and  self-monitoring). 

Implications:  Motivational 
Strategies  for  CBI 

Implications  for  strategies  promoting 
achievement  through  CBI  include  those 
derived  from  cognitive,  affective  and  social 
learning  theories.  Cognitive  support  can  be 
provided  to  the  learner  by  embedded 
learning  strategies  (Jonnasen,  1988). 
Affective  support  is  provided  in  through 
strategically  designed  effort,  strategy  and 
ability  attributional  feedback  and  by  giving 
learners  a  perception  of  control.  From  a 
social  learning  perspective,  feedback  can 
be  geared  towards  helping  achieve  self- 
efficacious  behavior;  cooperative  learning, 
peer  modelng  and  proximal  goal-setting  are 
suggested  for  boosting  motivation  through 
the  enhancement  of  self-efficacy. 


Cognitive  approach  to  motivation  in 
CBI 

CBI  can  enable  the  development  of 
self-regulated  behaviors  by  providing 
adequate  cognitive  support  through  the 
Incorporation  of  learning  strategies  into 
software  design.  Weinstein  and  Mayer 
(1986)  define  learning  strategies  as 
"behaviors  and  thoughts  that  a  learner 
engages  in  during  learning,  and  that  are 
intended  to  influence  the  learner's 
encoding  process"  (p.  315).  Jonassen 
(1988)  states  that  cunentty,  the  capabilities 
of  microcomputers  are  underutilized, 
resulting  in  shallow  processing  of  content. 
He  recommends  a  stronger  cognltivefy- 
oriented  approach,  making  teaming  more 
meaningful,  generative  and  engaging 
during  CBI.  According  to  Jonassen  (1988), 
generative  learning  strategies,  that  is,  those 
that  require  learners  to  consciously  relate 
new,  incoming  Information  to  existing 
schema,  can  be  effectively  embedded  in 
CBI  lessons.  Examples  of  such  strategies 
are  outlining,  creating  images,  cognitive 
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mapping,  paraphrasing,  summarizing.  Thus 
criteria)  tasks  (eg.,  practice  hems)  which 
require  studens  to  use  these  learning 
strategies  actively  must  be  incorporated. 
CBI  can  be  instrumental  In  furthering 
achievement  in  this  respect  by  replacing 
rote  recall  with  deeper  levels  of  processing, 
by  forcing  students  to  demonstrate 
strategic  behavior  before  they  can  advance 
further  in  the  lesson,  and  facilitating 
students'  tasks  by  performing  mechanical 
functions  for  them. 

With  their  unique  attributes, 
computers  also  aUow  facilitation  of  cognition 
through  a  model ng  of  cognitive  functions 
(Claik  &  Salomon,  1986).  For  example, 
processing  cues-  eg.,  highlighting  can  help 
learners  who  cannot  use  these  effectively 
(Jonassen  &  l-tanuum,  1987).  Gilingham 
(1988)  proposes  that  In  addition  to 
presenting  text  in  meaningful  ways,  CBI  can 
monitor  the  reading  of  text  in  various  ways 
to  increase  reader  comprehension. 

The  rote  of  another  important 
determinant  of  achievement  and 
motivation-  "metacognWon"  is  being 
increasingly  recognized  (eg.,  Armbruster, 
1983;  Baker  &  Brown,  1983;  Flaveli, 
1979).  MetacognttJon  is  described  as  the 
"knowledge  and  control  the  child  has  over 
his  or  her  own  thinking  and  learning 
activities."  (Baker  4  Brown,  1983.  p.  353). 
Use  of  cognitive  and  metacognitivo 
strategies  can  strengthen  the  effectiveness 
of  primary  strategy  use,  promote 
performance  and  perception  of  control. 
Jonassen  refers  to  these  as  "support 
strategies",  or  higher  level  executive 
control,  and  suggests  that  self-monitoring 
advisement  or  prompting  on  activities  like 
planning,  attending,  encoding,  reviewing, 
evaluating  can  be  efficaciously  incorporated 
in  CBI. 

Motivational  CBI  and  the  social 
learning  perspective 

Soda)  learning  theories  such  as 
Bandura  (1977)  and  Schunk  (1982;  1984) 
have  empirically  demonstrated  me  benefits 
of  interventions  designed  to  improve  self- 
efficacy  and  hence  performance  among 
tow-achievers.  Among  those  which  are 
relevant  to  CBI  are  effort  attributjonal 
feedback  related  to  past  aocomplshments, 
ability  feedback,  proximal  goal-setting  and 
peer  modeing. 


Research  on  cooperative  learning 
approaches  with  CBI  is  beginning  to  identify 
variables  critical  for  improved  performance 
(Johnson,  Johnson  &Stanne,  1985; 
Hooper  &  Hannafin,  1988;  Hooper,  Ward, 
Hannafin  and  Clark,  1989;  Webb,  1984) 
and  fomenting  positive  attitudes  (Datton  & 
Hannafin,  1984).  Johnson  &  Johnson 
(1985)  found  that  a  combination  of 
cooperative  teaming  and  computer- 
assisted  instruction  has  an  especially 
positive  impact  on  female  students'  attitude 
towards  computers.  The  role  of  the  peer 
group  as  one  of  the  most  important  external 
sources  of  support  in  computer-based 
instruction  is  emphasized  by  Brown  (1986). 
In  Ight  of  these  findings,  CBI  should  be 
designed  to  be  adaptable  for  use  with 
groups.  Teachers  must  also  explore 
benefits  of  peer  modeling  approaches 
(Bandura,  1982)  to  improve  self -efficacy  via 
cooperative  teaming.  Software  design 
should  be  flexible  so  that  it  includes  goal 
structures  (Johnson  &  Johnson,  1985), 
indtees  for  group  and  individual  success 
and  accountability.  Prompts  on  proximal 
goal-setting,  and  abiHty  attributional 
feedback  based  on  past  successes  can  be 
powerful  mediators  in  improving  self- 
efficacy,  leading  to  increased  persistence  in 
task  performance  (Schunk,  1989).  The 
importance  of  performance  feedback 
cannot  be  overemphasized:  if  it  is  frequent, 
contingent,  informative  and  encouraging,  it 
has  primary  impact  on  performance 
attributions  and  self-efficacy  (Corno  & 
Rohrkemper,  1984). 

Motivational  strategies  from  the 
affectlvo  domain 

Learner  Control 

Researchers  of  CBI  beleve  that  giving 
some  kind  of  control  to  learners-  the  nature 
of  which  is  stl  elusive-  is  desirabie:  It 
increases  learners'  feelings  of  perceived 
control  and  self-efficacy  (Bandura,  1982; 
Stipek  and  Weisz,  1981 ),  makes  teaming 
more  relevant  to  the  needs  of  learners,  and 
improves  attitudes  (Weisz  &  Cameron, 
1989).  Lepper  &  Hodnti  (1989)  claim  that  a 
lack  of  control  can  lead  to  teamed 
helplessness.  According  to  Merrill  (1987), 
learners  themselves  arrive  at  self- 
determined  instructional  strategies  which 
are  optimum,  when  given  an  opportunity  to 
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exercise  choice  over  them.  In  light  of  such 
benefits,  it  is  unfortunate  that  students' 
autonomy  and  choice  decline  as  they 
advance  in  grade  level  (Ecles  &  Mkkjeley, 
1989).  CBI  can  be  instrumental  in 
compensating  for  the  structural  Inflexibilty 
of  school  systems. 

Which  learners  benefit  from  learner 
control  In  improving  motivation  is 
determined  by  relationships  among  ability, 
prior  knowledge,  task  difficulty  and  nature 
of  control  (Snow,  1980;  Steinberg,  1977). 
Interactions  among  these  variables  are  the 
subject  of  currrent  research  on  learner 
control.  However,  given  their  Intrinsic 
flexibiity,  computers  offer  instructional 
designers  a  myriad  of  possibilties  in 
presenting  instructional  content. 
Incorporation  of  strategies  that  allow 
le-,.  .»ers  to  choose  amount,  sequence, 
,jace,  difficulty  level  of  content  can  be 
achieved  relatively  easily.  However,  all 
strategies  should  be  considered  with 
attention  to  empirical  findings  to  date  (eg., 
Hannafin,  1984). 

ThflfteteandWhisttes'ofCBI 

The  instantaneous  image  conjured  of 
motivation  in  the  realm  of  computer-based 
instruction  -  complex,  arresting  animation, 
sounds,  colorful  graphics  and  text, 
although  simpistic  in  nature,  has  piqued 
the  curiosity  of  researchers  who  have 
studied  these  media  attributes  individually 
or  collectively. 

The  embellishments  that  computers 
offer  can  be  highly  effective  in  promoting 
extrinsic  motivation.  Surber  and  Leeder 
(1988)  required  fourth  and  fifth  graders  to 
complete  two  versions  of  a  lesson,  one  with 
graphic  feedback,  and  the  other  with  text 
feedback.  Students  in  the  graphic 
feedback  condition  returned  to  the  r* 
more  often  than  those  in  the  text  feeujack 
condition.  The  advanced  visual  capabiWes 
of  computers-  graphics  and  animation, 
permit  the  creation  of  effective,  credible 
fantasies,  an  important  variable  identified  in 
the  design  of  computer  games  (Malone, 
1981).  Competition  and  challenge  can  be 
provided  with  a  score  and  time  counter,  and 
by  adapttvftty  varying  task  dHficulty 
(Jonassen  wd  Hanuum,  1987).  Recently, 
studies  wht  h  compare  the  motivational  and 
achieveme  it  influences  exerted  by  an 
embellished  versus  a  simple,  non- 


embellshed  version  of  a  lesson  are 
beginning  to  show  that  the  former  not  only 
attract  children  more  often,  but  students 
also  demonstrate  increases  in  achievement 
scores  over  time  (Lepper  and  Hode  , 
1989).  Thus  the  cumulative  effects  of 
entities  such  as  color,  graphics,  and 
animation  can  be  instructionaUy  and 
motivattonalfy  powerful. 

Fftort  Ahllrtv  and  Sjffllflgy,  attrihnfonal 

lfifidbafik 

The  objective  of  attributional 
feedback  Is  to  facilitate  learners'  own 
perception  of  themselves  as  able, 
efficacious  learners.  It  should  help  them 
see  that  effort  and  effective  strategy  use 
improve  performance  (Brophy,  1983),  and 
hence  foment  a  change  in  negative 
orientations  towards  temsefves  (McCombs, 
1984).  Research  has  demonstrated  the 
benefits  of  ef  ,i  attributional  training  in 
shifting  learners'  dysfunctional  beliefs 
about  their  own  abifty  (uncontrollable 
factor),  towards  attributions  to  effort  and 
strategy  use  (Dweck,  1975).  Clifford  (1984) 
believes  that  strategy  attributional  feedback 
on  failures  can  change  them  into  problem- 
solving  situations,  diverting  attention  from 
the  agative  aspects  of  failure.  Studies 
employing  strategy  training  have  yielded 
successful  results:  in  one  such  study  by 
Kurtz  and  Borkowski  (1984),  students  who 
attributed  success  to  effort  were  more 
strategic  than  those  attributing  outcomes  to 
abiity.  Although  no  prescriptive  rules  exist 
for  the  design  of  effort  versus  ability 
attributional  feedback,  relationships 
between  aptitudes,  attributional  style,  age 
and  sex  are  significant  in  determining  the 
situation  specific  requirements  of  such 
feedback. 

Researchers'  efforts  at  changing 
students'  negative  perceptions  about  their 
own  abiity  have  focused  on  shifting  such 
belief s  towards  effort  and  strategy 
attributions,  with  the  assumption  that 
student  behavior  and  attitudes  are 
malleable  (Brophy,  1985).  Although  such 
research  has  yielded  positive  results, 
perceptions  obout  one's  abiity  (an 
uncontrollable,  stable  causal  dimension) 
exert  a  more  powerful  influence  in 
predicting  performance,  affect  and  behavior 
than  effort,  a  controllable  factor  (Nicholls, 
1979;  Werner,  1984).  Thus  both  abiity  and 
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effort  warrant  consideration  in  the  design  of 
feedback.  Students  like  to  perceive 
themselves  as  both  able  and  hardworking 
(Covington  &  OmeRch,  1979).  Brophy 
(1983)  states  this  succintJy:  "Students  must 
be  able  to  conclude  that  they  have  the 
ability  to  meet  the  demands  made  on  them, 
if  they  make  reasonable  efforts  to  do  so." 
(p.  202).  To  accomplish  such  perceptions 
the  difficulty  level  of  tasks  should  be  easy  to 
medkjm,  with  varying  abilty  and  effort 
attributions,  depending  on  student  ability, 
nature  of  task,  demonstrated  persistence. 

Indvidual  differences  such  as  age 
appear  to  have  a  bearing  on  the 
conceptions  of  abilty.  NfchoHs  (1984)  has 
demonstrated  that  (Afferent  conceptions  of 
ability  should  be  used  for  different 
situations.  For  example,  for  adolescents 
the  concept  of  abilty  is  norm-referenced, 
whereas  for  young  children  it  is  self- 
referenced.  Young  children  associate 
effort  with  abilty,  whereas  adolescents 
attrUxneNghabiltiytoaciffkxjIttaskwhich 
requires  less  effort.  Thus  excessive  effort 
attribuUonal  feedback  may  be 
counterproductive  to  adolescents' 
conception  of  abilty,  and  beneficial  for 
young  children's  beliefs. 

Conclusion 

Academic  learning  is  intrinsically 
difficult,  requiring  concentration,  self- 
regulated  behaviors,  persistence,  cognitive 
effort  and  a  positive,  mastery-oriented 
outlook.  In  this  context,  the  role  of 
motivational  influences  In  improving  affect 
and  achievement  becomes  even  more 
critical.  The  motivational  role  of  computers 
transcends  beyond  providing  short-term 
interest  and  excitement  for  supplementary 
classroom  tasks.  As  classroom  and  home 
use  of  computers  increases,  an  innovative, 
creative  role  must  be  delegated  to  CBI  in 
the  development  and  maintenance  of 
intrintste  motivation.  From  a  cognitive 
viewpoint,  CBJ  must  provide  meaningful, 
cognftivety  supportive  instruction,  geared 
towards  improved  performance;  from  an 
affective  perspective,  CBI  should  captivate 
the  learner's  attention,  promote  feelngs 
and  expectations  of  success,  improve 
perception  of  control,  increase  positive 
attributions  to  effort  and  abilty;  and  finally, 
from  a  social  learning  perspective,  CBI  must 
enhance  self-efficacy  and  foster 


achievement  and  positive  affect  through 
social  Interactions,  proximal  goaf-setting 
and  attributional  feedback.  The  concept  of 
motivation  In  this  framework  is  construed  as 
that  In  which  students  show  moderate 
levels  of  arousal;  demonstrate 
metacognitive  awareness  and  persistence 
using  varying  learning  strategies  (computer- 
provided  or  self-generated);  make 
appropriate  attributions  to  abilty  and  effort; 
demonstrate  perception  of  control,  and 
learn  through  social  interactions. 
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Distance  education  and  learners'  individual  differences:  An  examination  of  different 
instructional  procedures  designed  to  accommodate  the  learning  characteristics  of  field- 
dependent  and  field  independent  learners. 

Learning  is  a  complex  process.  In  educational  settings  learning  is  the  product  of 
the  interaction  of  teacher,  student  and  instructional  content  and  the  environment  (Keefe, 
<987).  Each  of  these  complex  elements  has  wide  variation  and  that  variation  allows  for 
accommodations  to  the  teaching  and  learning  process,  Each  of  these  aspects  contributes 
something  to  the  existing  knowledge  of  learning  and  teaching.  Even  though  each  of 
these  has  been  extensively  investigated,  much  remains  to  be  discovered  about  the 
learning  process  and  how  it  can  be  optimized. 

Due  to  the  noncontiguous  nature  of  distance  education  with  its  physical 
separation  of  teacher  and  students  (Keegan,  1980)  certain  issues  common  to  many 
educational  contexts  become  exaggerated  and  therefore  provide  unique  opportunities 
for  examination  (Calvert,  1988).  This  separation  of  teacher  and  students  leads  to 
concerns  regarding  the  interpersonal  relationships  between  the  teacher  and  the 
students  and  the  effect  on  cognitive  achievement.  Perhaps  singling  out  one  of  these 
characteristics  and  examining  it  in  a  specific  context,  the  distance  education  classroom, 
would  have  practical  applications  for  learning  and  instructional  theory  for  other  areas  of 
educational  concern.  Characteristic  differences  in  cognition  shape  individual  learning  and 
performance  and  therefore  have  implications  for  teaching  and  learning. 

It  has  been  suggested  by  some  educators  that  individual  differences  are  a  valid 
focal  point  for  school  reform  and  restructuring.  In  a  learner-centered  school  setting  there 
is  apt  to  be  attention  given  to  individual  learning  styles  (Keefe,  1987).  Evidence  can  be 
found  in  recent  research  that  the  individual  differences  in  learning  style  that  the  learner 
brings  to  any  learning  oituation  is  worthy  of  examination  (Ausbum  &  Ausbum,  1978; 
Burger,  1985;  DeBasio,  1986;  Halpin  &  Peterson,  1986;  Harris,  1987,  in  Calvert,  1988; 
Moore  &  Bedient,  1986;  Moore  &  Dwyer,  1991;  Wieseman  &  Portis,  1990).  Individual 
differences  do  make  a  difference! 

Learning  Styles 

The  concept  of  learning  style  is  a  basis  for  understanding  the  ways  students  learn 
(Keefe,  1987).  Knowledge  of  cognitive  styles,  one  aspect  of  total  learning  style, 
influences  how  students  learn,  how  teachers  teach,  and  how  teachers  and  students 
interact  (Kogan,  1971;  Witkin,  Moore,  Goodenough,  &Cox,  1977).  Understanding 
student  learning  leads  to  consideration  of  the  teaching  and  learning  environment  and  that 
proceeds  logically  to  educational  improvement.  The  value  o\  including  lear    i  styles  in 
the  instructional  design  of  school  curricula  is  that  information  about  learning  styles 
provides  a  more  extensive  profile  of  the  learner  beyond  information  that  comes  from 
knowledge  of  the  student's  IQ,  aptitude  test  scores,  and  grade  average  (Messick,  1984; 
Keefe,  1987).  In  this  manner  educators  can  get  a  comprehensive  assessment  of  the 
student  and  this  is  one  of  the  important  first  steps  in  the  instructional  design  process 
(Dick  &  Carey,  1985;  Gagne,  1985). 

Learning  style  refers  to  the  way  people  absorb,  process,  and  retain  information 
(DeBello,  1990).    Everyone  has  a  preferred  learning  style  and  that  style  makes  the  same 
teaching  method  effective  for  some  and  ineffective  for  others  (Dunn,  Beaudry  &  Klavas, 
1989).  Learning  styles  are  thought  to  be  pervasive  and  consistent  indicators  of  personal 
preferences.  According  to  the  National  Association  of  Secondary  School  Principles 
learning  styles  are  the  "characteristic  cognitive,  affective,  and  physiological  behaviors 
that  serve  as  relatively  stable  indicators  of  how  learners  perceive,  interact  with,  and 
respond  to  th  learning  environment'1  (Keefe,  1987).  Cognitive  styles  are  information- 
processing  hauits;  affective  styles,  motivational^  based  processes;  physiological  styles, 
biologically-based  responses.  Learning  styles  have  also  been  separated  into 
environmental,  social,  emotional,  and  physical  categories.  (Dunn  &  Dunn,  1978). 
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Cognitive  styles  are  information  processing  habits  a  learner  uses  to  perceive, 
think,  problem  solve  and  remember  (Messick,  1984).  The  cognitive  style  characterizes 
the  person  and  continues  to  do  so  in  a  way  that  is  stable  over  time  (Manning,  in  Wapner  & 
Demick,  1991).  It  was  out  of  the  individual-differences  branch  of  c<>gnitive-style  research 
that  the  study  of  perception  focused  on  what  came  to  be  known  as  cognitive  styles  or 
cognitive  controls  (Witkin  &  Goodenough,  1981).  The  term  cognitive  came  to  be  used 
because  the  style  is  pervasive  across  both  perceptual  and  intellectual  activities. 

A  careful  search  of  the  literature  reveals  more  than  20  dimensions  of  cognitive 
style  to  be  found  in  experimental  research.  These  can  be  further  categorized  into 
reception  or  concept  formation  styles.  Some  of  the  reception  styles  are:  perceptual 
mode  preferences,  field  independence/dependence,  scanning,  constricted  vs.  flexible 
control,  tolerance  for  incongruity,  automatization,  conceptual  vs.  perceptual  styles.  The 
concept  formation  styles  include:  conceptual  tempo,  conceptualizing  styles, 
categorizing,  complexity  vs.  simplicity,  and  leveling  vs.  sharponing  (Keefe,  1987). 

The  other  two  aspects  of  learning  styles  are  the  affective  and  physiological 
dimensions.  The  affective  dimension  includes  these  constructs:  curiosity,  persistence, 
anxiety,  frustration,  locus  of  control,  risk-taking,  self-efficacy,  and  self-actualization.  Styles 
related  to  affective  emotional  aspects  take  into  account  an  individual's  motivation,  self- 
confidence,  persistence,  need  for  structure  and  guidelines,  and  autonomy  and  seif- 
directedness. 

The  third  dimension,  the  physiological  styles:  include  biologically  ba^d  aspects 
such  as:  timeof-day,  sex-related  behavior,  need  for  mobility,  and  environmental  factors 
(Keefe,  1987).  Included  here  are  the  visual,  auditory,  tactile  and  kinesthetic  styles  familiar 
to  most  people.  Styles  related  to  environmental  elements  refer  to  an  individual's  comfort 
level  with  the  surroundings,  such  as  the  lighting  level  of  the  room,  how  much  noisu  can 
be  tolerated  or  is  preferred,  and  ergonomic  considerations  like  seating  comfort. 

Field  Dependence 

Of  all  the  above  mentioned  cognitive  styles  one  has  many  fuaures  and 
characteristics  that  make  it  especially  suitable  to  oe  singled  out  for  this  study:  field 
dependence/independence.  There  are  several  reasons  for  this  suitability.  The  cognitive 
style  of  field  dependence  is  probably  the  most  researched  cognitive  style  which  gives  a 
strong  foundation  upon  which  to  draw  conclusions.  The  great  research  activity  in  field 
dependence  and  independence  can  be  attributed  in  part  to  its  breadth  and  to  its  real, 
visible  manifestations  in  everyday  life.  One  of  the  outcomes  of  the  research  on  field 
dependence  over  the  years  is  the  theoretical  framework  for  recognizing  the 
connectedness  of  psychological  phenomena  and  functions  previously  not  thought  to  be 
related  (Witkin  &  Goodenough,  1981).  The  evolution  and  growth  of  the  science  of 
educational  psychology  is  ari  ongoing,  dynamic  process.  In  this  process  new  evidence 
emerges  which  is  added  to  the  present  body  of  knowledge  so  that  new  principles  am 
formed  and  new  theories  are  offered.  In  this  way  phenomena  which  were  observed  many 
years  ago  take  on  fresh  perspectives  when  they  are  linked  to  previously  unrelated 
contexts.  In  addition  to  these  reasons  there  exist  effective  and  relatively  easy  methods  to 
assess  the  presence  of  field  dependence  (Witkin  &  Goodenough,  1981)  Field 
dependence  is  a  measure  of  a  cognitive  style  approach  which  brings  an  individual 
differences  perspective  to  the  experimental  study  of  social  interaction  (Bertini, 
Pizzamiglio,  &  Wapner,  1986). 

The  concept  of  field  dependence  evolved  from  research  during  World  War  II 
which  was  searching  for  a  way  to  determine  pilots'  aptitude  for  visual  and  spatial 
orientations.  In  those  days  the  way  field  dependence  was  measured  war>  to  plane  the 
subject  in  a  chair  that  was  tilted  relative  to  gravity  in  a  darkened  room,  in  front  ot  the  chair 
was  a  rectangular  frame  and  a  cylindrical  rod.  The  frame  was  also  reoriented  so  that  i?  war. 
out  of  alignment  with  gravity.  The  subject  was  to  align  the  rod  up  to  his  pnrcfipJion  of 
uprightness.  Whether  the  subject  could  align  the  rod  so  that  it  way  upright  r&oartiloss  of 
its  relativity  to  the  frame  or  whether  the  subject  would  align  the  rod  with  !h$  frame)  wSich 
appeared  to  be  upright  is  indicative  of  the  subject's  ability  to  orient  hy'wa!  ot,jaci\*.  Vhc 
field  dependent  person  relies  more  on  tho  visual  environment  to  make  a  judgment  *s  to 
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thu  vertk  ality  of  an  object  whereas  the  field  independent  person  orients  the  rod  to  a 
vertical  position  regardless  of  ihv  outward  apporirance  of  the  rod  and  frame  (Wapner,  S.  & 
Demick,  J. 1991 ;  Wilkin,  K,  Moore,  C.  A.,  Goodenough,  D.  R.,  &  Cox,  P.W.,1977). 

Researchers  noticed  that  subjects  have  a  preferred  way  of  integrating  the  many 
sources  of  information  from  th<  environment  that  contribute  to  cognitive  decisions, 
Differences  among  subjects1  preferences  emerged,  And  they  noticed  that  people  tend 
to  i  >b  that  same  frame  of  reference  consistently  to  integrate  information  across  a 
multitude  of  tasks  (Witkin  &  Goodenough,  1981).  This  ability  was  later  related  to  an  ability 
to  separata  an  item  from  an  organized  field  which  led  to  both  the  definition  of  field 
dependence  and  to  other  methods  of  measurement.  Yearu  of  experimental  research 
attempts  have  yielded  tho  conclusion  that  these  modes  of  esiublishi  j  ti*  upright 
depends  on  rolurral  either  to  the  body's  position  or  to  the  external  field.  Hence  the 
designation  ot  fMd  dependence  for  those  who  refer  to  the  external  visual  field  to 
determine  uprighi  and  field  independence  fo  those  who  refer  to  their  body  position, 

Held  dependence!  is  a  f  apacity  to  overcome  oi  analyze  embedded  contexts  in 
perceptual  functioning.  Field  dependent  and  field  independent  si.jjocts  differ  in  how 
they  perceivu  th  »  difference  and  not  in  ho*  accurately  they  perceive.  The  into:  t  of 
Witkin's  original  research  was  that  field  dependence  an*1  independence  exist  along  a 
continuum  That  is,  there  is  no  clear  cut  labelling;  subjectb  only  dei  lonstrate  more  or  less 
fielr!  dependence.  At  the  field  dependent  end  of  the  continu  m  si  Ejects  exhibit  global 
behaviors.  The  less  field  dependent  end  of  the  continuum  mairfests  behaviors  that  are 
specific  ,  nd  articulate 

Subjects  who  are  classitied  as  field  dependent  rely  more  rx  visual  cues  in  order 
o  make  decisions  about  the  environment.  They  a»o  more  socially  wanted,  attentive  to 
interpersonal  cues,  have  a  preference  to  being  physically  close  to  others  expmss 
emotional  openness  in  communication  with  others,  rely  more  on  information  that  is 
perceived  to  come  from  the  world  of  objects  and  people  around  them.  They  depend  on 
eternal  referonts  to  achieve  solutions.  They  t  ave  greater  interpersonal  competencies; 
are  more  at  home  with  people  because  they  need  other  people  tc  pro  'ide  them  v*\\h 
standards  (in  ambinuous  situathns)  for  judgment  and   ;tioM.  They  are  likely  t<  have 
professional  lives  involving  the  h  manitibs,  u<e  social  science*,  and  the  helpi  -i 
professions  (Bertini,  Pizzamiglio,  &  Wapner,  1986).  They  alu,  iond  to  be  influenced  hy 
autfioritat  ve  opinion.  Furthermore  field  dependent  poopk  rn,<v  need  detailed 
insti  notions  fiom  the  teacher  in  problem  giving  stratey^s  and  explicit  o/planutions  oi 
performance  outcomes  (Witkin,  1977). 

Subjects  classified  as  field  independent  nuv<  p  'nore  «  struct,  ^»peri, ->nal 
orientation.  Thoy  demonstrate  greatoi  t  iiysical  and  an,  >tional  distanninc,  They  unction 
with  a  greater  degree  of  autonomy  in  inter-pe'sonal  behavior,  They  w  /internal 
referents  to  restructure  the  universe.  They  p*e»yr  u-'ativelv  impersor  *l  situations  and 
maintain  greater  psychological  and  phonal  space  fr  >m  fliers  (Grenne  1^76).  It* /are 
less  !:Ke!y  to  require  detailed  explanations  a.  J  more  like!/  t<  work  with  leus  *j  Jdanca  than 
am  field  dependent  student,    hey  show  gsjatei  s«Ms  than  tho  ..eld  deoendeK.  tau  <*cts 
in  cognitive  restructuring  They  tend  to  nhi/ose  vocations  ii  \hh  schna-n  *nd 
mathematics  and  favor  isolation,  uiteilectualizatton  and  projection  over  social  integration  u< 
information. 

In  summary  «i  personality  theory  has  svoiv^  from      >os<    ationt  ano  f.-jdic  'i« 
of  ?be  psy<:nology  of  1  ild  (japonduncn  'ield  dopni  <Jence'ir'  tapencMJC'  »?  «*  process 
variable  describing  ways  o*  orienting  and  u  ^unn'mg  a  d  ,i<.t  th<  >  -caj:;s  th  ai  r'ny 
uoaio,  it  describes  pervasive  <t  tensions  ui  individual  f>-   ^>nint,  r  *  sell  cm*.  jt*n.  i  1 
behavior  ano  is  prcdictat^  across  bitu.itions:  it*  table  ovei  w  v  nr,  not  ine 
absence  of  tfv/:  other.  Instead  it    \  ip- m.  "ith  on*:  nun  wing  0*'  lino'  M<*4ti>.  * 
uompot-  nci^s  than  the  other,  *<m  p<  n  ha.  "lahik*  liuu  (fy,  p^opU*  &  «pt  'inUar 
particular  circurnutanccr  n  is  vaiim  ■•4utu« 
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Research  Background  of  Field  Dependence 

The  construct  of  field  dependence  is  net  related  to  some  other  constructs  which 
at  first  seem  to  be  related.  For  instance  numerous  studios  have  found  field  dependence 
is  not  related  to  the  attitudinal  construct  of  locus  of  control  (Witkin,  H.  &  Goodenough, 
6.1981)    Field  dependence  is  a  process  variable  describing  the  individual's  ability  in 
information  assimilating  processes  to  separate  self  ano  non-self.  Locus  of  control  is  an 
attitudinal  or  belief  variable  representing  the  individual's  perception  of  the  amount  of 
personal  or  external  control  over  life  results.  Field  dependence  has  also  been  found  to 
be  unrelated  to  the  cognitive  styles  of  *>rtraversion/introversion  and  impulsivity/reflexivity 
(Roodin,  Broughton,  &  Vaught,  1974;  fiHapson,  19^;  Tobacyk,  Broughton,  &  Vaught, 
1975,  in  Witkin  &  Goodenough,  1981,  p.  48) 

The  theory  asserts  that  the  outcome  of  teaming  doos  not  differ  for  the  two  types 
of  learners  but  the  procedure  used  in  their  learning  may  differ  (Wapner  &  Dei  nick,  1991) 
This  theoretical  position  raises  questions  relevant  to  the  educational  implications  of  lel\ 
dependence.  These  questions  address  the  issues  of  how  field  dependence  is  <  elated  to 
academic  achievement  and  how  different  instructional  procedures  accommodate  the 
individual  learning  differences  between  the  two  types  of  learners.  A  common  approach  to 
adapting  instructional  strategies  to  the  individual  Earner  employs  aptitude-treatment 
interaction  (ATI)  (Cronbach  &  Snow,  1969).  In  this  method  the  teacher  uses  differnnt 
strategies  for  each  type  of  learner  to  see  if  there  is  an  interaction  between  the  learner's 
aptitude  and  the  treatment.  The  instructional  strategy  is  designed  to  complement  the 
learner's  preferred  method  of  learning.  The  goal  of  ATI  research  is  to  identity  which 
learners  will  learn  beat  when  taught  one  way,  and  which  learners  possessing  an  opposite 
trait  may  learn  best  in  a  different  way.  Such  an  investigation  may  reveal  the  importance  o' 
matching  students'  learning  styles  and  instructional  methods  (Jacobs  &  Gedeon,  1982). 

While  previous  research  has  yielded  conflicting  and  inconclusive  results,  some 
important  and  substantive  findings  have  emerged.  Some  studies  have  reported  no 
differences  in  students'  achievement  for  their  cognitive  st\  e,  the  treatments  they 
received  or  any  interaction  of  the  two  (Canino  &  Cicchelli,  1988;  Grippen  &  Ohmacht, 
1977;  MacNeil,  1980;  Njus,  Huges  &  Stout,  1981). 

Other  studies  found  that  field  independent  students  achieved  higher  scores 
than  field  dependent  students  no  matter  which  treatment  tr  ey  received  (Carrier, 
Davidson,  Higson,  &  Peterson,  1984;  Carrier,  Joseph,  Krey,  &  LaCroix,  1983;  Fraiit 
1984;  Halpin  &  Peterson,  1986;  Kiewra  &  Frank,  1988;  Provost  1981;  Smith  &  Standal, 
1981;  Strawitz,  1984a,  1984b;  Thornell,  1977;  Threadgill,  1979) 

Some  studies  found  significant  interactions  with  field  dependence  and 
instructional  treatments  (Abraham,  1985;  Douglass  &  Kahk\  1978;  Kiewm  A  Frank, 
1986;  McLeod  &  Adams,  1970;  McLeod,  Carpenter  McCormaok,  &  Skvariciu^,  1978; 
Satterly  &  Telfer,  1979;  Threadgill-Sowder  &  Sowder,  1982).  In  the  study  by  Satterly  A 
Telfer,  (1979),  the  interaction  reflected  diffe*  mces  in  achievement  between  .ognitive 
style  groups  for  one  treatment  or  differences  in  achievement  between  treatments  for  one 
cognitive  style  group 

The  study  by  Abraham  (1985)  it  the  only  one  which  found  a  diboHi*  V 
interaction  that  showed  equal  leve's  of  achievement  foi  H»<j  cognitive  style  groups 
receiving  different  instructional  treatment;    In  this  study  foreigi ,  'urionts  learning  to  use 
participle  phrases  in  either  a  deductive  or  example-instructional  tream.^t  achieved  best 
when  their  cognitive  style  was  matched  with  the  ii  -tnjctional  treatment  bout  buited  for  that 
style,  Other  studies  which  claim  disordinal  interaction  (Uoey'ans  &  Kahle,  1978;  McLeod 
&  Adams.  1979;  McLucxJ,  Carpenter,  McCormack,  &  Skvancius,  '  V8;  Threadgill-ttowder 
&  Sowdar,  1982)  s^em  V  show  that  when  th  *re  was  significant  interaction  the  field 
independent  student  respond  more  to  variation,  n  ins* actional  manipulations, 

Ar;  shown  from  the  above  research  results  it  is  difficult  fo  reach  definitive 
conclusions,  The  insti  actional  treatments  ir  many  of  the  studies  Veined  the  level  of 
structure  l  correspond  to  the  theoretical  notion  that  field  dependent  students  pmfei  inl 
achieve  beut  in  a  more  structured  learning  environing:  and  field  independent  students 
ii*  efei  and  achievn  best  in  a  less  structured  learning  envirounwit.  Results  do  not  show 
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this  to  be  true  in  all  cases.  Often  the  effects  of  the  cognitive  style  overcomes  the 
instructional  treatment  and  one  cognitive  style  performs  best  regardless  of  the 
instructional  treatment.  Much  needed  research  may  bring  more  conclusive  results. 

Distance  Education  Research 

Research  in  distance  education  in  the  60's  and  70*s  was  mainly 
concerned  with  management,  administrative  and  global  descriptive  theory 
building  (Coldeway,  1988;  Holmberg,  1990;  Moore,  1985).  With  the  increased 
demand  in  educational  needs  for  adult  education  and  the  addition  of  new 
communication  technologies  educators  became  interested  in  building  and 
maintaining  enrollments.  Much  of  the  research  was  aimed  at  the  practical, 
organizational  aspects  of  distance  learning  in  the  world.  In  the  last  two  decades  a 
new  trend  in  distance  learning  research  has  appeared:  the  application  of 
scientific  research  methods  to  attempts  to  investigate  the  commonalities  of 
existing  educational  research  with  the  emerging  field  of  distance  education 
research  (Coldeway,  1988).  Also,  theorists  are  beginning  to  contribute  to  a 
theory  of  distance  education  (Moore,  1973  &  1977  in  Holmberg,  1990).  With  this 
new  emphasis  the  importance  of  methodologies  of  research  emerges.  In 
distance  learning  the  instructional  content  is  conveyed  from  teacher  to  student 
through  some  medium  such  as  satellite,  telephone  lines,  microwave,  computer 
connections  and  regular  mail.  With  the  evolution  of  new  technologies  distance 
education  has  grown  and  changed  from  the  early  days  of  correspondence  study 
that  once  ider  tified  distance  learning,  There  are  many  more  opportunities  to 
design  and  use  interactive  strategies  in  the  instructional  curriculum  of  distance 
education  than  there  once  was.  Since  modern  distance  education  is  a  departure 
from  the  norm  as  far  as  educational  contexts  go.  it  offers  an  opportunity  to  apply 
research  questions  to  a  different  environment.  In  this  way  the  body  of 
knowledge  can  be  expanded  by  including  a  nontraditional  context  in  the  research 
literature. 

Characteristics  of  learners,  such  as  the  need  for  interpersonal  contact,  the  ability 
to  work  in  an  unstructured  environment  and  satisfaction  with  the  educational  experience, 
characterize  success  in  the  distance  education  classroom  and  are  learner  attributes  that 
are  often  studied  in  traditional  classrooms  (Riddle,  1992).  Other  important  aspects  of 
individual  preferences  worthy  of  investigation  are:  self-directedness  and  autonomy,  need 
for  a  structured  learning  environment,  and  comfort  with  and  preferences  for  working 
individually.  Perhaps  by  singling  these  issues  out  and  examining  them  in  an  environment 
where  their  presence  is  emphasized  inferences  can  be  made  that  generalize  to  other 
educational  settings. 

Design  of  the  Study 

A  study  will  be  conducted  to  determine  i  the  learning  style  of  field 
dependence/independence  has  any  effect  on  ihe  cognitive  outcomes  and  attitudes  of 
students  in  a  distance  education  class.  This  research  study  is  an  examination  of 
characteristics  of  learners  and  the  kinds  of  instructional  strategies  used.  This  study  will 
compare  the  instructional  strategies  of  group  and  individual  learning  to  see  which  strategy 
results  in  greater  learning  and  which  strategy  students  prefe.  snd  enjoy  most  when 
learning  at  a  distance.  Quantitative  data  will  be  gathered  from  two  sources:  a  criterion 
referenced  test  of  the  instructional  content  presented  during  the  study  and  from  a  Likert 
scale  measuring  attitudes,  satisfaction,  and  preferences  toward  instructional  strategies  in 
i  distance  learning  class.  Both  of  these  tests  will  be  given  before  and  after  the  study  to 
ass'*5s  any  change  in  learning  and  attitude. 

Subjects  will  be  volunteer  college  undergraduate  students.  The  pool  of  potential 
volunteers  will  be  235  students  n  four  sections  of  an  undergraduate  class,  (a  section  is 
;  -i  same  class  taught  at  different  times).  The  subjects  will  be  tested  early  in  the  semester 
buor<*  the  study  begins  with  th  ee  instruments,  Tho  first  is  the  the  Group  Embedded 
F:igurus  Test  (GEFT)  given  to  determine  their  level  of  field  dependence.  The  (GEFf)  is  a 


9 

ERLC 


630 

684 


6 

Learning  styles  and  distance  education 


paper  poncil  test  composed  of  simple  figures  hidden  within  a  field  of  complex  figures. 
The  timed  tost  contains  complex  figures  divided  into  two  5-minute  testing  sessions.  One 
of  eight  simple  forms  is  embedded  in  each  complex  figure.  The  subject  is  asked  to  trace 
only  one  simple  form  in  pencil  over  the  lines  of  the  test  booklet.  The  purpose  of  the 
(GEFl*)  is  to  assess  an  individual's  perceptual  differentiation.  A  person  with  greater 
differentiation  typically  scores  above  the  median  (9)  and  a  person  of  lesser  differentiation 
typically  scores  below  the  median  (Witkin  ,  1971).  The  split-half  reliability  coefficient  of  the 
GEFT  of  332  subjects  with  the  Spearman-Brown  correction  was  estimated  at  0.89 
(Burkhalter  &  Schaer,  1985). 

The  second  test  will  be  the  criterion-referenced  multiple  choice  pretest  which  is  a 
test  of  their  mastery  of  the  content  presented  during  the  study.  The  test  will  be 
developed  and  then  reviewed  by  subject  matter  experts,  college  professors  who  teach  at 
a  distance.  The  third  test  will  be  the  Likert  Scale  which  is  a  self-report  instrument 
assessing  their  attitude  and  satisfaction  about  learning  in  groups  or  individually,  being  an 
active  participant  in  their  learning,  their  comfort  level  with  beirg  separated  from  the 
teacher,  and  their  confidence  in  receiving  the  same  instruction  at  a  distance  as  they  would 
have  in  a  traditional  class.  This  scale  was  developed  and  pilot  tested  by  this  researcher  on 
198  college  students  subjects  and  found  to  have  a  reliability  coefficient  of  0.75.  (This  test 
was  developed  using  the  expertise  of  an  evaluation  and  measurement  expert.  This  test 
was  also  reviewed  by  subject  matter  experts,  college  faculty  members  who  have  expertise 
in  distance  learning.) 

At  the  beginning  of  the  semester  the  students  will  be  randomly  assigned  to  one 
of  two  treatment  groups  by  dividing  the  class  role  into  two  halves.  The  first  treatment 
group  will  receive  the  instruction  in  smal!  groups.  A  small  group  will  consist  of  four 
students.  The  small  group  will  function  as  autonomously  as  possible.  They  will  be  given 
a  packet  of  instructions  which  will  be  designed  to  be  self-explanatory.  The  teacher  will 
operate  as  a  facilitator  only,  answering  questions  as  needed  but  encouraging  the 
students  to  work  together  to  reach  the  stated  objectives.  The  second  group  will  receive 
the  same  instructional  materials  and  objectives  as  the  small  group  treatment  receives. 
This  group  will  work  on  the  materials  individually.  The  teacher  will  facilitate  and  answer 
questions  for  students  as  they  arise.  The  lesson  plan,  the  objectives  and  the  teacher  will 
be  the  same  for  botf  groups.  The  difference  will  be  that  the  students  in  the  group  will  be 
able  to  work  together  to  accomplish  the  objectives  of  the  instructional  module,  and  the 
students  who  work  individually  will  not  receive  assistance  from  their  fellow  students.  The 
instruction  will  take  place  over  2-3  class  periods. 

All  of  the  instruction  will  be  delivered  at  a  distance  by  using  two-way  video  and 
two-way  audio  media  so  that  students  and  teacher  can  see  and  hear  each  other.  At  the 
end  of  the  sessions  all  students  will  take  the  same  test,  that  was  given  before  the  start  of 
the  study.  In  addition  to  the  criterion-referenced  test  students  will  complete  the  Likert 
scale  instrument  again. 

Research  Questions 

This  study  will  examine  the  following  research  questions: 

1  Field  dependent  students  will  show  greater  gains  from  the  achievement  pretest  to  the 
achievement  posttest  test  in  the  group  treatment  than  in  the  individual  treatment. 

2.  Field  independent  students  will  show  greater  gains  from  the  achievement  pretest  to 
the  achievement  posttest  test  in  the  individual  treatment  than  in  in  the  small  group 
treatment. 

3.  Field  dependent  students  report  that  they  prefer  group  learning  instruction  and 
actually  achieve  better  in  the  group  learning  treatment. 

4.  Field  independent  students  report  that  they  prefer  individual  learning  and  actually 
achieve  better  in  the  individual  learning  treatment. 
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This  study  will  examine  the  following  hypotheses: 

There  is  no  significant  difference  in  the  change  scores  of  field  dependent  and  field 
independent  students  from  the  criterion-referenced  oretest  to  the  criterion-referenced 
posttest. 

There  is  no  significant  difference  in  the  change  scores  of  field  dependent  and  field 
independent  students  on  the  pretest  Likert  scale  assessing  preferences  for  instructional 
strategy,  need  for  interpersonal  interaction  and  confidence  with  being  in  a  distance 
learning  class  and  the  posttest  Likert  scale. 

The  differences  in  means  for  the  four  groups,  field  dependent  in  the  small  group 
treatment,  field  dependent  in  the  individual  treatment,  field  independent  in  the  small 
group  treatment,  and  field  independent  in  the  individual  treatment,  will  be  analyzed  for 
any  significant  differences  by  using  an  analysis  of  variance.  This  research  is  ongoing  and 
as  such  data  is  not  available  about  the  results  of  the  study.  Data  from  this  research  will  be 
available  to  interested  parties  upon  request. 

Implications  for  Distance  Education  and  Traditional  Education 

In  distance  education  contexts,  much  research  on  individual  difference  factors 
needs  to  be  done  (Calvert,  1988;  Coldeway,  1988).  Some  benefits  of  learning  style 
research  specifically  for  distance  education  include  guiding  the  goals  of  the  distance 
learning  class,  predicting  the  success  of  learners,  allowing  for  selection  of  learners, 
designing  courses  more  specifically  for  learner  needs,  having  indicators  as  to  the  type 
and  quantity  of  interaction  between  teacher  and  students,  and  offering  learners 
opportunities  to  use  strategies  other  than  their  preferred  strategies  to  become  more 
versatile  in  their  learning  behaviors  (Ehrman,  1990;  Thompson  &  Knox,  1987). 

Knowledge  of  students1  cognitive  styles  not  only  has  relevance  about  individual 
learning  but  also  about  the  nature  of  teacher-pupil  interactions  and  of  social  behavior  in  the 
classroom  (Messick,  1984).  Educational  research  has  shown  that  cognitive  styles 
influence  how  students  learn,  how  teachers  teach,  and  how  teachers  and  students 
infract  (Witkin,  Goodenough,  &  Cox,  1977).  The  knowledge  of  cognitive  styles  has 
value  for  education  by  identifying  personal  characteristics  that  may  interact  with 
instructional  treatments  that  mediate  learning,  retention  and  transfer;  as  diagnostic  tools 
for  prescribing  interventions;  and  by  promoting  the  learner's  flexibility  (Messick,  1984). 
Results  of  research  on  individual  difference  have  relevant  applications  to  other  learning 
settings  so  long  as  practitioners  remain  aware  of  the  interdependence  of  individual 
differences,  social  and  environmental  factors,  and  learning  outcomes. 

Understanding  the  importance  of  learning  styles  to  education  leads  to  an 
expectation  of  the  impact  they  have  on  the  teaching/learning  environment.  Cognitive 
styles  might  provide  an  opportunity  to  tailor  the  mode  of  presentation  to  the  characteristics 
of  the  student;  improve  the  communication  between  teacher  and  student  when  teachers 
become  aware  of  student's  style  variations;  expand  student  thinking  and  learning 
strategies;  extend  educational  goals  to  uiclude  not  only  knowledge  acquisition  but  also 
student's  manner  of  thinking  (Messick,  1984).  Scores  on  an  achievement  test  don't  give 
enough  information  about  a  student.  More  information  is  needed.  Investigate  is  into  how 
students  learn  can  provide  even  more  information  for  the  total  picture  of  the 
teaching/learning  environment. 
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Two  design  features  of  the  computer  distinguish  it 
from  other  educational  media  —  greater  display  capa- 
bilities and  the  ability  to  provide  continuous  adaptive 
feedback.  Relevant  and  sustained  interactivity  is  central 
to  instructional  design  (Gagn6, 1985;  Jonassen,  1988; 
Wager  &  Wager,  1985).  The  computer  affords  a  wide 
range  of  interactive  strategies  at  all  levels  of  learning. 
Interactive  learning  strategies  are  usually  grouped  under 
the  heading  of  practice.  Practice  that  is  designed  to 
promote  higher-level  processing  (Salisbury,  1988)  often 
encourages  students  to  use  cognitive  strategies  such  as 
informal  hypothesis-testing  to  solve  problems  or  resolve 
conceptual  conflicts.  Of  the  many  available  computer- 
based  instruction  (CBI)  formats,  simulations  offer  the 
most  potential  of  exploiting  both  the  display  and  interac- 
tive strengths  of  computers. 

Simulations  are  constructed  for  two  primary  pur- 
poses: to  study  relationships  in  a  model  of  a  system  (e.g. 
scientific  simulations);  and  to  teach  users  about  these 
relationships  (Duchastel,  1990-91).  As  an  instructional 
tool,  computer  simulations  allow  users  to  have  meaning- 
ful experiences  within  a  system  or  domain  through 
experimentation  and  discovery.  Simulations  can  be 
designed  to  represent  only  a  portion  of  the  real  world, 
thereby  making  an  abstract  situation  more  concrete. 
Users  can  generate  and  test  hypotheses  in  a  simulated 
environment  by  examining  changes  in  the  environment 
based  on  their  input,  but  without  the  potentially  undesir- 
able consequences  of  reality.  Thus,  simulations  help  to 
enable  students  to  learn  specific  problem  solving  strate- 
gies by  receiving  immediate  feedback  on  decisions  made 
while  in  the  structured  environment  of  the  simulation 
(Gress,  1982).  In  instructional  settings  the  student  inter- 
acts with  the  simulation  and  observes  the  computer 
output  as  either  verbal  or  visual  feedback  (VernerJcy  & 
Osin,  1991).  Feedback  can  also  be  presented  in  real  time, 
accelerated  time,  or  decelerated  time.  This  can  be  espe- 
cially useful  when  the  simulated  event  cither  occurs  too 
fast  (e.g.  an  internal  combustion  engine)  or  too  slow  (e,g, 
deforestation)  in  real  life  for  the  feedback  to  have  any 
meaning. 

Computer-based  microworlds,  which  simulate  infor- 
rr^iion  and  relationships  of  complex  systems  in  areas 


such  as  mathematics,  science,  and  geography,  also  offer 
promise  in  the  design  of  interactive  learning  environ- 
ments (Collins,  Adams,  &  Pew,  1978;  diSessa,  1982; 
Papert,  1981;  White,  1984;  Montague,  1988). 
Microworlds,  aconcept derived  from  constructivism,  are 
small  representations  of  content  areas  or  domains 
(Forman  &  Pufall,  1988).  Microworlds  are  often  con- 
fused with  simulations.  Although  microworlds  and 
simulations  can  remair  entirely  distinct,  the  characteris- 
tics of  the  two  can  overlap  or  even  be  synonymous 
depending  on  their  design  and  on  how  each  is  used  in  a 
learning  interaction.  A  microworld  is  often  defined  as  a 
representation  of  the  simplest  model  of  a  domain  that  can 
be  recognized  by  an  expert  and,  most  importantly,  which 
also  matches  the  user's  abilities,  interests,  and  experi- 
ences. 

Simulations,  on  the  other  hand,  are  generally  de- 
signed to  mimic  real  life  experiences,  such  as  a  flight 
simulator.  When  designed  as  microworlds,  simulations 
allow  learning  contexts  and  situations  to  be  created 
which  extend  beyond  the  walls  of  a  classroom  —  stu- 
dents can  try  to  survive  a  trip  to  the  South  Pole,  save  a 
beached  whale,  run  a  campaign  for  President  of  the 
United  States,  or  explore  the  systems  of  the  human  body. 
Simulations,  especially  those  with  game-like  features, 
offer  the  potential  for  intrinsically  motivating  activities 
(Malone,  1981;  Rieber,  1991).  For  simplicity,  we  will 
use  the  term  "simulation"  throughout  this  paper  to  refer 
to  learning  environments  which  incorporate  features  of 
microworlds. 

The  instructional  effectiveness  of  a  simulation  is 
determined  by  three  major  aspects  of  its  design:  the 
scenario,  the  underlying  model,  and  the  instructional 
overlay.  The  scenario  recreates  a  real  life  situation  and 
determines  what  happens  and  how  it  takes  place.  The 
model  is  usually  a  mathematical  formula  that  reflects  the 
causal  relationships  tliat  govern  the  situation.  The  in- 
structional overlay  includes  the  instructional  design  fea- 
tures of  a  simulation  and  how  they  should  be  used  to 
optimize  instruction.  Simulators  also  enhance  the  trans- 
fer of  learning  by  teaching  complex  tasks  in  an  environ- 
ment that  approximates  the  real  world  setting  (Reigeluth 
&  Schwartz,  1989).  Simulations  provide  students  with 
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opportunities  to  study  processes,  procedures,  and  phe- 
nomena that  either  cannot  be  taught  under  any  circum- 
stances or  cannot  be  easil  y  taught  using  traditional  meth- 
ods (Hannafin  &  Peck,  1988).  Simulations  differ  from 
interactive  tutorials,  which  help  students  learn  by  provid- 
ing information  and  using  appropriate  question-answer 
techniques.  In  a  simulation  the  student  learns  by  per- 
forming the  activities  to  be  learned  in  a  context  that  is 
similar  to  the  real  world  (Alessi  &  Trollip,  198S). 

Simulations  can  perform  many  instructional  func- 
tions. They  are  frequently  used  as  follow-up  practice 
strategies  for  higher-level  learning  during  traditional 
instructional  designs.  This  type  of  approach  is  based  on 
a  deductive  learning  model,  where  students  are  formally 
introduced  to  a  concept  or  a  principle,  followed  by 
practice.  A  deductive  approach  provides  structure  to 
guide  a  student  during  an  exploration  of  the  simulated 
environment  in  predetermined  steps,  thus  allowing  for 
directed  practice.  Recent  studies  have  reported  success- 
ful applications  of  structured  simulations  as  practice 
activities  in  the  teaching  of  physical  science  (Rieber, 
1990a,  Rieber,  1990b;  Rieber,  Boyce,  &  Assad,  1990). 
One  of  the  most  important  components  of  these  struc- 
tured simulations  has  been  the  use  of  animation  as  visual 
feedback.  Animation  can  provide  a  constant  stream  of 
visual  feedback  based  on  moment-to-moment  student 
input. 

In  addition,  simulations  can  also  be  used  in  inductive 
learning  strategies.  Inductive  approaches  to  learning 
have  students  interact  with  instances  of  subject  matter 
before  generalities  are  introduced  and  explained.  An 
inductive  approach  is  based  on  letting  students  discover 
the  structure  of  the  subject  matter  on  their  own  (Bruner, 
1966;  Slavin,  1988).  The  same  simulation  used  as 
practice  in  a  deductive  approach  can  also  give  a  student 
the  opportunity  to  freely  explore  an  environment  and 
learn  by  discovery  or  through  game-like  contexts.  Do 
students  need  the  added  structure  of  guided  activities 
called  for  by  deductive  approaches,  or  can  they  learn  by 
just  being  given  the  opportunity  to  interact  freely  with  a 
rich  model  of  a  learning  domain?  There  is  a  need  to 
continue  the  research  of  animation-based  interactive 
strategies  on  the  computer  beyond  their  use  as  traditional 
practice  activities. 

The  purpose  of  this  study  was  to  investigate  the 
effects  of  interactive  learning  strategies,  specifically 
visually-based,  real-time  computer  simulations,  on  in- 
ductive learning  of  adults  in  physical  science. 

SUBJECTS 

The  subjects  consisted  of  1 60  uppcrclass  undergradu- 
ate students  (juniors  and  seniors)  who  were  enrolled  in  an 
introductory  computer  education  class.  Participation 
was  voluntary  with  extra  credit  in  the  course  provided  as 


an  additional  incentive  to  participate. 

MATERIALS 

The  CBI  content  of  the  tutorial  and  simulation  activi- 
ties involved  the  physics  principles  of  Newtonian  me- 
chanics. The  tutorial  was  divided  into  four  parts.  The 
first  part  reviewed  fundamental  background  vocabulary 
and  concepts,  such  as  mass,  weight,  inertia,  and  force. 
This  part  also  formally  presented  Newton's  first  law  of 
motion.  The  second  part  introduced  Newton's  second 
law  of  motion  and  then  described  applications  of 
Newton's  first  and  second  laws  given  equal  forces  in 
opposite  directions  in  one-dimensional  space.  The  third 
part  extended  the  application  of  Newton's  first  and 
second  laws  to  include  the  effects  of  unequal  forces  in 
one-dimensional  space.  The  fourth  pan  introduced  and 
explained  applications  of  these  principles  in  two-dimen- 
sional space.  All  instruction  was  presented  at  an  intro- 
ductory level  intended  for  novices.  The  lesson  was  de- 
signed to  exclude  as  much  formal  mathematics  as  pos- 
sible and  instead  concentrated  on  concept  formation  and 
application. 

DEPENDENT  MEASURES 

Three  data  sources  were  studied:  performance,  as 
measured  by  student  scores  on  a  posttest;  comprehension 
monitoring;  and  response  confidence.  The  posttest  con- 
sisted of  27  rule-using  questions  using  a  multiple-choice 
format  (1  answer  and  4  distractors).  The  posttest  mea- 
sured student  performance  on  the  four  lesson  objectives 
previously  described  above.  KR-20  reliability  of  all  27 
questions  was  .86. 

At  various  points  during  the  instructional  treatment, 
students  were  prompted  to  answer  the  following  ques- 
tion: "If  you  were  given  the  posttest  right  now,  what  do 
you  think  your  score  would  be?"  Students  answered  from 
0%  to  90%,  in  multiples  of  10%.  These  introspection 
data  on  students'  perceived  level  of  learning  during  the 
lesson  were  designed  to  be  an  informal  assessment  of 
their  comprehension  monitoring.  For  this  analysis,  com- 
prehension monitoring  is  operationally  defined  as  the 
students'  answer  to  this  question  immediately  before  the 
administration  of  the  posttest  Students  were  also  asked 
this  question  before  they  began  the  lesson,  in  order  to  test 
whether  or  not  the  groups  were  similar  in  the  perception 
of  how  much  they  already  knew  about  the  material.  No 
significant  differences  were  found  among  the  groups  at 
the  start  of  the  experiment,  F(5, 154)=  1.70,  /?>.!, 
A/Se=4.31.  On  average,  students  nelt  they  would  get 
about  38%  of  the  posttest  questions  correct  if  given  the 
test  immediately. 

Response  confidence  during  the  posttest  was  also 
collected.  Immediately  after  answering  each  question, 
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students  were  asked  to  rate  how  confident  they  were  in  Tukey's  studentized  range  test  for  follow-up  multiple- 

their  answer  on  a  Likert-type  scale  from  1  to  9  where  1  comparisons  on  means, 
was  "I  guessed"  and  9  was  "I  know  I'm  right/* 

RESULTS  AND  DISCUSSION 

PROCEDURES 

A  significant  interaction  was  found  between  Tutorial 
All  instruction  and  testing  was  administered  by  com-  and  Simulation,  F(2,154)=7.13,/k:.01,  A/Se=279.89,  on 
puter.  Subjects  were  randomly  assigned  to  one  of  the  posttest  performance,  as  illustrated  in  Figure  3.  Students 
treatment  groups  as  they  reported  to  the  computer  lab.  in  the  Structured  Simulation/No  Tutorial  condition 
Once  instructed  to  begin,  subjects  completed  the  com-  (mean=74,5%)  performed  as  well  on  the  posttest  as  any 
puter  lesson  individually.  The  posttest  was  automati-  of  the  three  Tutorial  groups  (mean=83.0%).  All  of  these 
cally  administered  to  each  student  upon  completion  of  four  conditions  performed  significantly  better  than  stu- 
the  four  lesson  parts.  Approximately  90  minutes  were  dents  in  the  control  condition  (No  Tutorial/No  Simula- 
needed  to  complete  the  lessons  in  each  experiment.  tion)  (mean=54.1%).  Student  scores  in  the  Unstructured 

Simulation/No  Tutorial  condition  (mean=67.6%)  did  not 

DESIGN  vary  significantly  from  those  of  the  control  condition. 

These  performance  results  support  the  hypothesis  that 

A  2  X  3  factorial  design  was  used  in  this  study.  Two  adult  students  would  be  able  to  induce  and  apply  relevant 

levels  of  Tutorial  (Yes,  No)  were  crossed  with  three  physical  science  rules  given  experiences  with  real-time 

levels  of  Simulation  (Structured,  Unstructured,  None),  computer  simulations.  However,  the  simulation  was 

The  respective  simulation  activity  was  provided  immc-  only  effective  in  promoting  inductive  learning  when  it 

diately  after  each  of  the  four  lesson  parts  in  the  tutorial,  was  structured  so  as  to  introduce  a  limited  number  of 

Deductive  learning  strategies  were  comprised  of  condi-  learning  variables  at  a  time.  Learning  was  obviated  when 

tions  which  included  the  tutorial.  When  paired  with  the  this  structure  was  removed  from  the  simulation  and 

tutorial,  each  simulation  condition  acted  as  follow-up  subjects  were  placed  in  a  simple,  yet  open-ended  simu- 

practice  in  the  tradition  of  direct  instructional  methods,  lation.  It  should  be  noted  that  the  tutorial  appeared  to  be 

Inductive  learning  strategies  were  comprised  of  the  two  an  especially  potent  treatment  variable  and  probably 

simulation  conditions  presented  as  the  sole  learning  masked  the  effects  of  the  simulation  activities  when  they 

experience.  The  no  simulation,  no  tutorial  condition  were  paired  with  it. 

acted  as  a  posttest-only  control.  A  significant  interaction  was  found  between  Tutorial 

The  structured  simulation  condition  consisted  of  a  and  Simulation,  F(2,154)=4.99,  /k.01,  AfSe=3.15,  on 

series  of  four  separate  computer  simulations  in  which  comprehension  monitoring,  as  illustrated  in  Figure  4. 

students  were  given  increasing  levels  of  control  over  a  Follow-up  analyses  indicated  two  groups.  Those  who 

simulated,  free-floating  object  (called  a  "starship").  This  were  given  the  tutorial  or  the  structured  simulation  felt 

activity  is  termed  "structured"  because  each  simulation  that  they  had  learned  significantly  more  than  students 

presented  a  controlled  number  of  new  subskills  to  stu-  given  only  the  unstructured  simulation  or  no  instruction, 

dents.  Each  successive  simulation  incorporated  the  Given  only  the  unstructured  simulation,  therefore,  stu- 

subskills  from  the  preceding  simulation.  dents  perceived  they  had  not  learned  anything  and  felt 

The  unstructured  simulation  condition  consisted  of  totally  unprepared  for  the  posttest.  However,  students 

an  open-ended  and  unstructured  simulation  in  which  given  only  the  structured  simulation,  but  no  tutorial,  felt 

students  assumed  full  control  over  the  animated  starship  as  though  they  had  learned  as  much  as  students  given  the 

from  the  very  beginning.  The  simulation  was  presented  traditional  tutorial. 

using  a  game-like  context.  Students  were  provided  with  The  analysis  of  students*  response  confidence  during 
the  goal  of  "docking"  their  animated  starship  with  a  the  posttest  also  indicated  a  significant  interaction  be- 
"spacc station"  (represented  by  a  circle  on  the  screen).  A  twecn  Tutorial  and  Simulation,  F(2,154>=4.53,  /k.05, 
scorekeeping  feature  was  built  into  the  game  in  order  to  M Se=2.77,  as  shown  in  Figure  5.  Follow-up  analyses 
provide  an  extra  level  of  challenge  as  well  as  to  help  indicated  three  distinct  groupings.  Subjects  in  all  three 
students  see  whether  or  not  their  skills  were  improving.  Tutorial  conditions  indicated  the  strongest  level  of  con- 
Student  were  given  a  total  of  10  trials  with  this  activity  fidence  in  their  answers  to  posttest  questions 
to  provide  a  similar  level  of  exposure  as  that  experienced  (mean=7.6).  Next  in  order  of  response  confidence  were 
by  students  in  (lie  structured  simulation  condition.  Fig-  subjects  in  the  Structured  Simulation/No  Tutorial  condi- 
ures  1  wd  2  illustrate  examples  of  the  structured  and  tion  (mcan=6.1).  Finally,  subjects  in  both  the  Unstruc- 
unstructured  simulation  activities.  tured  Simulation/No  Tutorial  condition  and  No  Simula- 
Statistical  procedures  included  analysis  of  variance  tion/No  Tutorial  control  condition  constituted  the  third 
(ANOVA)  on  each  of  the  dependent  measures  and  and  lowest  response  confidence  ratings  group 
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SHUTTLE  MISSION  TWO:  SPACE  DOCK 


Make  the  shuttle  come  to  a  complete  stop  inside  the 
circle. 

USE  THESE  KEYS: 

and  Spins  the  shuttle 

<SPACE  BAR>     Thrusts  the  shuttle  in  the  direction 
that  it's  pointing 

The  shuttle's  tail  must  be  in  the  circle. 


STATUS:  Moving 


SPEED  « 1 


FIGURE  1.  An  example  of  a  structured  simulation  activity. 


MISSION  GOAL:  FLY  THE  SHUTTLE 
TO  THE  SPACE  STATION 


A 


CONTROL  PANEL 


<SPACE  BAR>  for  thrust 


Spin 


H  Speed:  0    V  Speed:  3        Heading:  0 


Try:  4      Score  last  try:  470    Score:  494 


FIGURE  2.  An  example  of  an  unstructured  simulation  activity. 
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FIGURE  3.  Interaction  between  Tutor iat  and  Simulation  on  the  Performance  Data 


(mcans=4.7  and  4.4,  respectively).  This  pattern  is  simi- 
lar to  that  of  the  comprehension  monitoring  data  in  that 
students  given  only  the  structured  simulation  felt  more 
confident  than  students  given  only  the  unstructured 
simulation.  However,  students  given  only  the  structured 
simulation  obviously  lost  confidence  while  answering 
the  questions,  as  they  were  distinctively  less  confident 
than  students  given  the  tutorial. 

So,  while  students  in  the  Structured  Simulation/No 
Tutorial  condition  performed  similarly  to  students  who 
received  direct  instruction  via  the  tutorial,  the  former  did 
not  feel  as  confident  in  their  answers  to  specific  posttest 
questions  as  the  latter.  Hence,  a  lack  of  confidence  may 
be  among  the  consequences  of  providing  adult  subjects 
with  experiential  learning  approaches,  rather  than  direct 
instruction.  Follow-up  surveys  with  the  students  indi- 
cated that  they  felt  very  uncomfortable  with  the  unstruc- 
tured simulations — they  had  expected  and  wanted  more 
structure.  When  adults  are  engaged  in  what  they  perceive 
to  be  a  formal,  or  school-like,  learning  experience,  they 
expect  school-like  learning  environments  to  best  aid 
their  learning,  and  may  actually  resist  the  opportunity  to 
learn  by  discovery.  This  point  has  been  echoed  by  adult 
educators  (Seaman  &  Fellenz,  1989). 

A  follow-up  study  has  been  conducted  using  chil- 
dren, although  data  have  not,  as  yet,  been  analyzed.  It  is 
expected  that  children  may  be  more  receptive  to  learning 
from  open-ended  activities  which  offer  little  resem- 
blance to  "school-like"  tasks. 
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FIGURE  4.  Interaction  between  Tutorial  and  Simulation  on  comprehension  monitoring  data. 
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FIGURE  5.  Interaction  between  Tutorial  and  Simulation  on  the  response  confidence  data. 
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I.  Introduction 


The  teaching  skills  workshops  were  originally  developed  in  1980  to  train  graduate  assistants  to 
teach  college  classes.  Since  then  they  have  evolved  into  workshops  that  are  being  used  by  the 
faculty -mstrnctors^ 

successfully  modified  for  use  with  different  audiences  and  under  varying  time  constraints.  To  date, 
over  800  faculty  members  have  completed  our  workshops.  As  a  consequence  of  these  workshops,  CIS 
is  involved  every  semester  in  helping  faculty  to  improve  their  classes. 

II.  Description  of  the  Workshops 

The  workshops  are  a  series  of  1 0  90-minute  sessions  which  are  open  to  all  faculty,  graduate 
instructors,  and  stafif.  The  individual  sessions  are: 

1 .  Student-Teacher  Relationships  -  This  workshop  describes  the  most  common  student  complaints 
about  instructors  and  suggests  ways  that  instructors  can  improve  their  relationships  with  students 
Student  services  at  Purdue  are  discussed  and  directions  for  referring  students  with  personal  or 
academic  problems  are  included. 

2UniverSiryPoUcieSandProcc<lu,eSRelatedtoTeaching-PurduepoliciesandregulationS 
concerning  instruction  are  discussed  as  outlined  in  the  Academic  PiwedurM  Mim| 

the  FacultYjind  Staff  ffinrtlrooh,  and  UniYersltvRMHilirtnn*  a  R»frr.nVf  Rookfnt.'st„ripn,<, 

Staff,  and  Fatuity.  The  procedures  for  appealing  grades  at  Purdue  are  also  discussed. 

3.  Designinglnstruction  -  The  importance  of  having  consistent  learning  objectives,  instructional 
activities,  and  evaluation  procedures  is  discussed.  Practice  is  provided  in  developing  objectives 
and  evaluating  them  using  a  checklist.  Various  types  of  instructional  activities  are  presented 
Exercises  provide  practice  in  matching  objectives,  instructional  activities,  and  evaluation. 

4.  Lecturing  Techniq  ues  -  This  workshop  considers  the  ••trengths  and  limitations  of  lecturing  as  a 
teaching  method.  Ways  to  structure  a  lecture  and  utilize  basic  public  speaking  techniques  are 
discussed.  Techniques  and  checklists  for  evaluating  lectures  are  provided. 

5.  Audio-ViSualTwhidqueS--Thi^ 

rooms  and  audio-visual  equipment .  Recommendations  for  using  the  most  common  types 
audio-visual  equipment— 

audio  and  public  address  systems,  chalkboards,  charts  and  graphics,  computer-uisplayed 
matenals.film,  objects,  overhead  transparencies,  slides,  and  videotapes  —  are  given. 

e.Micro-TeachingExercise-Theworkshopprovideseachparticipantwiththeopporhmityto 
prepare  and  present  a  five-minute  segment  of  classroom  instruction.  The  presentation  will  be 
videotaped  and  played  back  for  the  presenter  and  a  small  group  of  peers.  During  the  playback,  an 
evaluation  form  will  be  completed  by  all  participants.  Following  this,  strengths  of  the 
presentation  and  suggestions  for  improvement  will  be  discussed. 

7.  Discussion  Techniques-  The  workshop  presents  the  advantages  and  disadvantages  of  group 
discussion  as  a  method  of  teaching.  It  provides  an  organizational  framework  for  effective 
discussions  and  suggestions  for  conducting  discussions.  It  also  outlines  the  responsibilities  of 
discussion  leaders  and  participants,  conducting  discussions.  It  also  outlines  the  responsibilities  of 
discussion  leaders  and  participants. 


9 

ERIC 


67»  645 


8.  Developing  Tests — The  purpose  s  of  and  procedures  for  writing  c  ollege  tests  are  explained .  The 
strengths  and  weaknesses  of  true/false,  multiple  choice,  short  answer,  and  essay  tests  are  presented. 
Specific  suggestions  for  writing  each  type  of  test  item  are  discussed.  Good  and  bad  examples  of 
test  items  are  included. 

9.  Scoring  Tests  and  Assigning  Grades — Norm-referenced  and  criterion-referenced  grading  systems 
are  compared  and  contrasted.  The  Center  for  Instructional  Services'  test  scoring  and  analysis 
service  is  explained  and  its  use  for  test  improvement  described.  Several  methods  of  assigning 
course  grades  are  discussed.  A  variety  of  factors  to  be  considered  in  assigning  grades  is  presented, 
and  the  distinction  between  grading  and  evaluation  is  explained. 

10.  Evaluation  Instruction — This  workshop  discusses  techniques  for  assessing  the  effectiveness  of 
instructional  materials,  the  instructional  environment,  and  the  instructor.  Techniques  for  peer,  self, 
and  informal  student  evaluation  are  explained.  Two  formal  methods  of  student  evaluation  —  Small 
Group  Instructi  onal  Diagnosis  (SGID)  and  the  Purdue  Instruct  or  and  Course  Apprai :  al  System 
(CAFETERIA)— are  described. 

These  workshops  are  presented  as  a  series  during  the  fallsemester,  before  the  spring  semester, 
during  the  spring  semester,  before  the  summer  session,  and  during  the  summer  session.  During  the 
regular  sessions  (fall,  spring,  and  summer),  the  sessions  are  presented  twice  a  week  during  a  five- week 
period.  Before  the  spring  and  summer  sessions,  the  workshops  are  presented  in  an  intensive  three-day 
format. 

Kach  workshop  is  structured  around  a  printed  study  guide  distributed  to  the  participants  before  the 
workshop.  The  workshops  are  designed  to  be  highly  interactive  and  to  demonstrate  ideas  and 
techniques  presented  in  the  study  guiH*.  The  optimum  number  of  participants  is  15.  With  le^s  than  12 
participants,  the  "roup  does  not  provide  enough  interactivity.  Withmorethan  18,each  member  of  the 
group  doesnot  get  a  chance  to  participate  during  each  session.  In  order  to  keep  the  workshops  moving 
and  to  provide  a  change  of  pace,  two  facilitators  from  the  CIS  staff  are  present  at  each  workshop. 

Assignments  are  used  to  provide  a  common  experience  base  for  participants  and  to  generate 
materials  which  can  be  discussed  during  the  workshops.  There  are  three  of  these  assignments: 

1 )  participants  are  asked  to  submit  questions  about  college  teaching  that  they  want  answered, 

2)  to  visit  a  class  of  another  instructor,  and  3)  write  objectives,  instructional  activities,  and  test  items. 

in.  Advertising  the  Workshops 

Four  primary  techniques  are  used  to  promote  the  workshops . 

1 .  CIS  Newsletter  -  This  newsletter  is  distributed  to  all  faculty  and  staff  five  times  each  year. 

It  includes  workshop  dates  in  each  issue  and  discusses  the  workshops  once  every  fall  in  a  special 
issue  that  provides  an  overview  of  CIS  services. 

2.  University  channel  on  the  local  TV  cable  system — Dates  of  workshops  are  listed  in  the  coming 
events  section.  In  addition,  periodic  descriptions  of  the  workshops  given  in  the  news  portion  of  the 
programming. 

3.  Direct  n;  ail    Announcements  are  sent  to  faculty  and  staff  about  every  six  months  depending  upon 
length  of  the  waiting  list.  These  announcements  are  responsible  for  recruiting  most  of  the  workshop 
attendees. 

4.  Word  of  Mouth — This  method  has  provided  the  best  promotion  possible,  but  it  is  very  difficult 

to  measure.  Interested  individuals  frequently  call  and  ask  about  the  workshops.  When  asking  about 
the  workshops,  they  often  tell  us  that  colleagueshave  told  them  about  how  helpful  the  workshops  had 

been. 
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IV.  Conductingthe  Workshops 

Each  workshop  is  structured  around  a  printed  study  guide  containing  a  set  of  objectives.  The 
participants  are  expected  to  have  read  the  study  guides  before  the  workshops.  In  order  to  ensure  that  all 
points  are  covered ,  a  "couinuiry  sheet"  is  prepared  for  use  by  the  facilitators  during  each  workshop 
(See  Figure  1).  These  sheets  are  not  rigid  schedules,  but  rather  flexible  guidelines  which  allow  the 
facilitators  to  respond  to  questions  from  participants  and  to  spontaneous  teaching  opportunities.  These 
sheets  also  provide  a  list  of  the  items  that  are  needed  during  the  workshop.  This  has  helped  us  to  make 
sure  that  we  have  all  the  materials  that  we  will  need  for  a  particular  workshop. 

The  participants  are  encouraged  to  ask  questions  and  to  share  teaching  experiences  with  other 
participants.  The  workshops  are  designed  to  encourage  interaction  and  dialogue  among  all  participants 
and  all  facilitators.  This  dialogue  has  been  very  helpful  in  allowing  the  participants  to  share  tbeir 
experiences -both  good  and  bad -what  worked  and  what  did  not.  This  helps  to  foster  a  climate  of 
collegial  sharing  of  experiences  rather  than  lecturing  by  the  master.  It  is  during  these  dialogues  that 
we  begin  to  develop  a  relationship  that  later  allows  us  to  provide  consulting  services  to  some  of  the 
participants. 

The  workshop  assignments  involve  participants  in  meaningful  instructional  activities.  Participants 
write  objectives,  design  instructional  activities,  construct  test  items,  and  share  techniques  for 
improving  student-teacher  interactions.  The  participants  also  observe  another  instructor  in  a  teaching 
situation  in  the  microteaching  exercise.  In  this  highly  rated  assignment,  each  participant  prepares  and 
presents  a  five-minute  excerpt  from  a  lesson  to  be  videotaped.  The  videotape  and  the  live  presentation 
are  critiqued  (orally  and  in  written  form)  by  the  other  participants  and  the  facilitators. 

V.  Evaluation  of  the  Workshops 

Five  methods  are  used  to  evaluate  the  workshops. 

1 .  Verbal  comments  b  v  the  participants  -  These  informal  comments,  made  during  and  after  each 
workshop,  provide  valuable  input  for  reviewing  the  individual  sessions. 

2  Session  evaluation  forms — These  evaluation  forms  are  completed  by  each  participant  at  the  end 
of  each  session,  these  forms  (See  Figure  2)  ask  for  an  overall  reaction  to  the  session  as  well  as 
specific  responses  as  to  likes  and  dislikes  about  the  session  and  suggestions  for  improvement. 
These  forms  have  provided  some  of  our  best  feedback. 

1  Coiuse  andlnstructionEvaluation— This  form  uses  Likert-type  items  with  statements  about 
the  instructors,  the  materials,  the  media,  and  the  content.  It  serves  two  purposes:  to  improve 
the  workshops  and  to  demonstrate  how  this  type  of  evaluation  can  be  used  in  courses. 

4  Small  Group  Instructional  Diagnosis  (SGID) — This  se  ssion  is  conducted  by  a  fac  ilitator  not 
involved  with  the  workshop.  The  participants  are  broken  into  small  groups  and  evaluate  the  overall 
workshop.  The  facilitator  provides  specific  feedback  to  the  workshop  staff  (See  Figure3). 

5.  Open-ended  rating  scale-The  participants  are  required  to  pick  which  session  they  liked  the  bestand 
which  the  least.  They  are  then  asked  to  tell  us  why  they  made  their  particular  chokes. 

Evaluation  is  an  on-going  process  with  the  first  two  methods  used  during  and  following  each 
session,  and  the  latter  three  techniques  implemented  near  the  end  of  the  series  of  workshops.  It  should 
also  be  emphasized  that  as  with  any  technique  we  use,  we  used  this  one  to  teach  as  well  as  to  evaluate. 
Thus,  we  have  the  participants  experience  as  well  as  use  an  evaluation  technique  in  the  expectation 
that  they  will  want  to  use  them  later  in  their  own  teaching. 
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VI.  Follow-up  of  the  Workshops 


As  a  result  of  the  rapport  established  between  the  facilitators  and  the  instructors,  the  participants 
often  calluponthe  college  teaching  workshop  staff  for  consultation  followingthe  workshops.  The 
consulting  may  result  in  an  answer  to  a  simple  question  or  a  referral  for  a  service .  The  facilitators 
often  provide  referrals  to  other  individuals  or  servk  as  on  campus.  In  other  cases,  the  participants  may 
need  a  major  review  and  revision  of  their  courses.  The  facilitators  have  provided  instructional  design 
assistance  as  requested. 

The  facilitators  are  available  to  visit  individual  classes  of  former  CTW  participants.  Following 
the  class,  the  facilitator  sits  down  with  the  instructor  to  review  strengths  and  weaknesses  and  to 
provide  suggestions  for  improvement.  The  facilitators  also  review  videotapes  of  former  participants' 
classes  at  their  request.  Again,  the  facilitators  review  their  findings  with  the  instructors. 

Workshop  participants  sometimes  also  request  SGID's  of  their  classes.  One  of  the  workshop 
facilitators  conducts  these  SGID'  s,  discusses  the  results  with  the  instructor,  and  may  suggest  how  the 
instructor  can  improve  any  problem  areas  uncovered  by  the  SGID. 

VH.  Review  of  the  Workshops 

The  series  of  workshops  is  reviewed  by  the  staff  three  times  each  time  they  are  offered.  First,  the 
staff  reviews  any  weaknesses  and  suggests  revisions  the  week  before  the  workshops  start.  At  this 
session,  staff  assignments  are  made  for  all  10  workshops. 

Secondly,  the  staff  meets  once  each  week  during  the  regular  workshops  to  review  the  previous 
week's  sessions  and  to  plan  any  changes  in  the  coming  week's  sessions. 

Finally,  after  the  workshops  are  completed,  the  staff  reviews  all  the  participant  evaluations.  The 
staff  discusses  any  changes  which  might  be  made  to  improve  individual  sessions  or  the  series  as  a 
whole.  Generally,  we  try  to  work  on  the  workshop  that  received  the  lowest  rating.  This  way,  we  are 
constantly  improving  the  workshops. 

VIII.  Modification  of  the  Workshops 

The  individual  workshops  have  been  modified  by  the  staff  to  meet  specific  needs  and  requests.  If  a 
participant  raises  a  specific  question  related  to  the  content  of  a  particular  session,  the  staff  can  modify 
that  session  to  address  that  question. 

The  length  of  sessions  can  be  increased  or  decreased  to  meet  specific  needs.  They  can  be 
shortenedto  50minutes  to  fit  the  standard  class  period  at  Purdue.  Generally  we  have  found  that  the  90 
minute  length  provides  the  best  length  time  to  allow  all  the  participants  to  participate  but  keep  the 
workshops  moving  along  a  an  interesting  pace .  Howe  ver,  we  will  shorten  them  to  fit  the  needs  of  a 
particular  situation.  Most  often,  we  delete  elements  from  a  workshop  to  achieve  the  shorter  time 
frame. 

The  staff  has  also  reformatted  and  reshaped  sessions  for  particular  audiences.  For  example, 
sessions  have  been  modified  for  specific  departments  or  specific  teaching  strategies  and  to  meet  the 
needs  of  foreign-born  teaching  assistants.  We  have  also  modified  the  workshops  for  presentation  to  the 
trainers  at  a  local  corporation. 
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IX,  Summary 

The  College  Teaching  Workshops  are  attended  by  faculty  as  well  as  graduate  teaching  assistants. 
In  fact,  the  majority  of  the  participants  have  been  faculty.  The  workshops  have  resulted  in  a  continuing 
stream  of  requests  from  former  participants  for  different  forms  of  consultation  to  he  lp  improve  their 
teaching  efficiency  and  effectiveness. 

Requests  continue  for  special  workshops  to  help  faculty  and  graduate  teaching  assistants  in  their 
teaching  and  professional  presentations.  These  requests  have  often  come  from  former  workshop 
participants.  Thus,  the  workshops  have  helped  to  generate  an  increasing  interest  in  the  improvement  of 
teaching  on  the  Purdue  campus. 
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1 .  write  Agcnd*  on  board  an  go  over  objectives  in  study  guide. 
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Figure  2:  Session  Evaluation  Form 


PRESENTATION  EVALUATION  FORM 

(ma.  k  one) 
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1  especially  liked- 
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Other  suggestions  and  comments  include. 


Figure  3 :  Small  Group  Instructional  Diagnosis  (SGID)  Form 


Group  Form  for  SG9D 


Instructor 


Course 


Number  of  studwnts  In  yroup 
1.  What  do  you  like  about  this  couise? 


Date 


2.  What  specific  suggestions  do  you  have 
for  changing  this  course? 
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Interactive  technologies  such  as  computer-based 
instruction,   interactive  video,  hypertext  systems,  and 
the  broad  category  of  interactive  multimedia  systems 
are  increasingly  making  an  impact  in  educational  and 
training  settings.     Many  teachers,  trainers,  decision- 
makers, community  members,  and  business  and  government 
leaders  contend  that  these  technologies  will  change  the 
face  of  education  and  training  (Ambron  &  Hooper,  1990, 
1988;   Interactive  Multimedia  Association;  Lambert  & 
Sallis,   1987;  Schwartz,   1987;   Schwier,   1987;  U.S. 
Congress,  OTA,   1986^.     Of  concern  is  the  claim  by  some 
that  educational  technologists  are  no  longer  the 
leaders  in  developing  these  technologies.     Some  are 
concerned  that  educational  technologists  are  being  left 
behind.     One  reason  may  be  that  technologists  have 
neglected  to  prove  the  value  of  conducting  evaluations 
of  these  technologies  and  thus  cannot  always  show  data 
to  prove  that  systematically  designing  technological 
innovations  for  education  makes  a  difference.  One 
common  feature  of  models  for  systematically  designing 
instructional  materials  is  that  draft  versions  of  these 
materials  be  tested  with  representative  learners  to 
ensure  that  the  materials  are  effective  (Andrews  and 
Goodson,   1979)  in  the  process  called  formative 
evaluation.     Other  terms,   including  developmental 
testing,  pilot  testing,   field  testing  and  validation, 
are  occasionally  used  for  this  process.     Most  designers 
also  draw  a  distinction  between  formative  and  summative 
evaluation.     Generally,   formative  evaluation  is 
conducted  for  the  purpose  of  improving  the 
instructional  program  through  revision  (Dick  &  Carey, 
1990;  Gagne,  Briggs,  &  Wager,   1988;  Morris  & 
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Fitzgibbon,   1978).     Summative  evaluation  is  usually 
conducted  to  determine  the  overall  value  of  a  program, 
such  as  to  make  a  "go"  or  "no  go"  decision  about  a 
completed  program,  often  comparing  it  with  other 
programs  or  approaches  (Dick  &  Carey,  1990;  Geis, 
1987).     The  focus  of  this  paper  is  on  formative 
evaluation  of  interactive  technologies . 

While  educational  technologists  acknowledge  that 
we  should  be  conducting  more  researcn  and  more  research 
on  formative  evaluation  specifically,  the  value  of 
formative  evaluation  in  enhancing  the  learning 
effectiveness  of  instruc/  ion  has  often  been  shown.  One 
example  is  the  recent  meta-analysis  conducted  by  Fuchs 
and  Fuchs  (1986).     These  researchers  analyzed  the 
results  of  21  studies  which  investigated  the  effects  of 
formative  evaluation  of  materials  developed  for  mildly 
handicapped  students.     They  found  that  systematic 
formative  evaluation  significantly  increased  students' 
achievement  when  students  used  the  resulting  materials. 

When  to  conduct  formative  evaluation  is  not  always 
clear  in  instructional  design  models.  Simplified 
graphics  used  to  illustrate  some  models  show  formative 
evaluation  being  conducted  using  an  almost-final  draft 
of  the  instruction  at  the  end  of  the  design  and 
development  process.     This  may  be  appropriate  for 
simple,  print-based  instruction,  however  most  models 
show  that  formative  evaluation  is  an  ongoing  process 
conducted  throughout  development.     For  example,  Dick  & 
Carey  (1990)   in  their  widely-used  model  recommend  that 
formative  evaluation  include  frequent  reviews  of 
materials,   several  1:1  tryouts  with  learners  who 
represent  several  segments  of  the  target  popu] at ion ,  at 
least  one  small-group  tryout ,  and  a  field  test  in  the 
actual  learning  setting.     On-going  formative  evaluation 
is  particularly  important  in  developing  costly  and 
labor-intensive  interactive  technology-based 
instruction.     In  fact,  many  small  but  critical  design 
aspects  of  the  interactive  instruction,  such  as  user- 
interface  features  like  icons,  menus,  and  navigational 
tools,  are  evaluated  continuously  in  small  segments  as 
the  project  evolves . 

While  it  is  generally  agreed  that  formative 
evaluation  of  instruction  developed  in  any  medium  is 
critical,   it  is  the  premise  of  this  paper  that  it  is 
even  more  crucial  when  the  instruction  is  to  be 
delivered  via  interactive  technologies,  such  as 
computers,   interactive  video  or  the  various  forms  of 
interactive  multimedia  systems.     For  example,  most 
design  models  for  developing  computer-based  instruction 
(Gagne,  Wager,  &  Rojas,   1981;  Hannafin  &  Peck,  1988; 
Smith  &  Boyce ,   1984)  as  well  as  interactive  video  for 
instruction  (Kearsley  &  Frost,   1985;  Savenye ,  1990) 
call  for  conducting  formative  evaluations.  There   is  a 
tendency,  however,   for  evaluation  to  be  neglected 
during  development  due  to  constraints  of  budget, 
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personnel,  and  scheduling  ( Brenneman,   1989).     This  is, 
unfortunately,  especially  true  in  large-scale 
technology-based  projects,   in  which  costs  are  already 
high.     Patterson  and  Bloch  (1987)  contend  that 
formative  evaluation  is  often  not  done  during 
development  of  interactive  instruction  using  computers. 
They  mention  that  one  reason  for  this  may  be  that 
decision-makers  in  education  and  industry  hold  a 
negative  attitude  toward  formative  evaluation .  These 
authors  contend  that  educational  technologists  should 
recognize  this  fact,  and  help  such  constituents  see  the 
value  of  formative  evaluation. 

At  the  same  time  as  formative  evaluation 
procedures ,  and  sometimes  systematic  ins ^ruct ional 
design  itself,   is  sometimes  assailed  by  developers  of 
interactive  instruction  who  have  backgrounds  other  than 
instructional  design,  there  are  increased  calls  by 
funding  sources  and  educators  to  use  "qualitative"  or 
"naturalistic"  research  methods  in  studying  school  and 
training  processes  and  projects  (Bosco,   1986;  Clark, 
1983).     These  methods  are  considered  alternatives  to 
more  traditional  approaches,   such  as  using  scores  on 
paper-and-pencil  achievement  tests  in  experimental  and 
quasi-experimental  comparisons  of  programs.     Sadly,  at 
times  producers  of  interactive  educational  materials 
have  responded  by  collecting  data  on  inappropriately 
small  samples  of  learners,  watching  learners  without 
first  determining  the  evaluation  questions,  collecting 
too  much  data  to  effectively  analyze  later,  and/or 
focusing  on  how  well  students  like  instructional 
programs,  rather  than  on  whether  students  learn  through 
the  programs. 

Similar  to  the  contentions  about  use  of  formative 
evaluation  during  development  of  interactive 
instruction,  researchers  have  noted  that  there  is 
little  research  being  conducted  on  formative  evaluation 
(Chinien,   1990;  Geis,   1987;  Patterson  &  Bloch,  1987). 
Thus  it  is  likely  that,  just  when  we  should  be 
improving  the  ways  we  conduct  formative  evaluations, 
especially  when  developing  interactive  instruction,  we 
are  not  conducting  the  research  necessary  to  develop 
and  test  these  improvements. 

The  purpose  of  this  paper  is  to  present  methods 
for  planning,  conducting,  and  using  the  results  of 
formative  evaluations  of  interactive  technology -based 
instruction.     The  focus  is  on  practical  considerations 
in  making  evaluation  decisions,  with  an  emphasis  on 
alternate  methods  in  formative  evaluation. 

Foundational  Assumptions  and  Biases  of  This  Paper 

Lest  the  reader*  believe  this  author  is  contending 
that  all  these  approaches  to  evaluation  are  new,  we 
need  only  look  at  Markle's  1989  "ancient  history  of 
formative  evaluation"  to  remind  ourselves  that  versions 
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of  these  processes  have  been  used  by  designers  for  many 
years,  although  often  informally-     Markle  contends  that 
even  in  the  early,  more  "behavioralist " ,   days  of 
instructional  design >  developers  listened  to  their 
learners,  watched  them  carefully,  and  humbly 
incorporated  what  learners  taught  them  into  their 
drafts  of  instructional  materials-     Similarly,  what 
recent  authors,  especially  computer  scientists,  are 
calling  testing  in  "software  engineering  (Chen  &  Shen, 
1989;   "prototype  evaluation"   (Smith  &  Wedman,  1988), 
"prototype  testing",   quality  assurance"   (McLean,  1989), 
or  "quality  control"   (Darabi  &  Dempsey,   1989-90)  is 
clearly  formative  evaluation  by  another  name. 

A  controversy  swirls  in  education  and  in  our  field 
regarding  the  relative  value  of  "quantitative"  and 
"qualitative"  investigations .     "Quantitative"  usually 
means  experimental  or  quasi-experimental  research 
studies;   in  evaluation,   these  studies  often  compare  one 
approach  or  technology  with  another.  Some 
technologists  have  called  for  the  abandonment  of 
quantitative  comparison  studies  ( cf .  Reeves ,   1986  )  , 
claiming  they  answer  the  wrong  questions  in  limited 
ways . 

Use  of  the  term  "qualitative"  research  is  less 
clear.     It  usually  refers  to  studies  using 
anthropological  methods  such  as  interviews  and 
observations  to  yield  less  numerical  descriptive  data. 
Unfortunately  the  resulting  studies  sometimes  employ 
less  than  sound  research  methods*  Such  studies  have 
given  the  term  "qualitative  research"  a  bad  name  in 
some  circles,  notably  among  those  who  are  strong 
advocates  of  the  sole  use  of  quantitative  methods. 

It  is  the  view  of  this  author  that  when  planning 
evaluations  of  interactive  technologies  the  debate  is 
not  useful.     Most  practical  educational  developers  have 
for  many  years  used  a  blend  of  quantitative  and 
qualitative  methods  in  evaluation  "Quantitatively", 
there  is,   for  example,  a  long  tradition  of  using 
pretests  and  posttests  to  compare  the  performance  of 
learners  in  a  control  group  with  those  who  have  used  a 
new  educational  technology  program.  "Qualitatively", 
evalu^tors  have  long  collected  attitude  data  using 
surveys,   interviews  and  sometimes  observations. 
Alternate  research  methods  allow  for  collecting  more 
types  of  qualitative  data  to  answer  the  new  questions 
which  emerge  in  evaluating  new  technologies. 

A  more  fruitful  approach  to  the  issue  of  which 
types  of  research  methods  to  use  is  to  select  whatever 
methods  are  appropriate  to  answer  the  particular 
evaluation  questions.     Such  an  approach  is  in  line  with 
the  recommendation  of  Clark  (1983)  that  we  reconsider 
our  study  of  media.     This  approach  is  also  similar  to 
the  ROPES  guidelines  developed  by  Hannafin  and  his 
associates  (Hannafin  &  Rieber,   1989;  Hooper  &  Hannafin, 
1988)  which  blend  the  best  of  behavioral ism  and 
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cognitivism  in  what  they  call  "applied  cognitivism" . 
Finally,  selecting  methods  based  on  questions  supports 
Driscoll's  (1991)  suggestion  that  we  select  overall 
research  paradigms  based  on  the  most  urgent  questions. 
Driscoll  adds  that  instructional  technology  is  a 
developing  science  in  which  "numerous  paradigms  may  vie 
for  acceptibility  and  dominance"  (p.  310). 

A  final  fundamental  bias  of  this  paper  is  that  the 
most  important  question  to  ask  during  formative 
evaluation  remains,   "How  well  did  the  learners  learn 
what  we  intended  them  to  learn."     This  paper  presents 
several  types  of  data  collection  and  analysis  methods 
to  answer  important  questions  which  emerge  when 
interactive  instructional  technlogies  are  involved, 
'et  if  we  answer  these  questions  and  neglect  whether 
students  learn  we  will  not  know  whether  the 
technological  innovation  has  any  value. 

Benefits  of  Considering  Alternate  Methods 
of  B'ormative  Evaluation 

While,   as  noted  earlier,   the  ideas  are  not 
strictly  new,  there  are  several  reasons  for  a  new  and 
deeper  look  at  formative  evaluation  when  interactive 
technologies  are  involved.     At  one  level  developing 
instruction  using  technologies  such  as  computers  adds 
complexity  to  what  can  go  wrong  and  what  needs  to  be 
attended  to,  because  there  are  hardware  and  software 
issues  involved  (Patterson  &  Bloch,  1987).  For 
example,   interactive  systems  are  often  multimedia 
systems,   so  formative  evaluation  questions  often 
include  how  efffective  graphics,  animations, 
photographs,  audio,   text  and  video  are  in  any  lesson 
segment . 

A  second  reason  a  new  look  at  formative  evaluation 
methods  is  warranted  is  that  interactive  technologies 
now  allow  developers  to  collect  data  about  learners  and 
learning  that  could  not  technologically  be  collected 
before.     We  can  thus  look  at  learning  in  new  ways,  and 
answer  questions  we  may  have  wanted  to  answer  before. 
For  example,  computer-based  lessons  can  be  programmed 
to  record  every  keypress  a  learner  makes.  Developers 
can  thus  determine  how  many  times  a  student  attempts  to 
answer  a  question,  what  choices  they  make,   and  what 
paths  they  follow  through  hypermedia-based  knowledge 
bases.     One  danger,   of  course,   is  that  developers  can 
become  "lost  in  data",  collecting  data  without  regard 
to  evaluation  questions  and  what  to  do  with  the  data. 

A  third  reason  for  study  of  formative  evaluation 
methods  is  that  with  the  recent  renewed  emphasis  on 
qualitative  research  in  education,  have  come  increased 
numbers  of  good  studies  using  alternate  research 
methods.     It  is  fortuitous  that  these  methods,  many 
borrowed  from  other  fields,  particularly  anthropology 
and  sociology,  are  being  tested  and  results  reported  at 
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a  time  when  developers  of  interactive  technologies  are 
looking  for  new  ways  to  measure  how  much  our 
technologies  help  learners  learn. 

A  final  reason  to  expand  our  views  of  evaluation 
methods  is  to  "push  the  envelope"  of  useful  knowledge 
in  our  own  field  of  educational  technology,  as  many 
researchers  and  developers  have  called  for.  Reigeluth 
(1989)  states  that  our  field  is  now  at  a  crossroads 
with  considerable  debate  taking  place  regarding  what  we 
should  study  and  how.      Winn  (1989)  calls  for 
researchers  to  conduct  descriptive  studies  yielding 
more  information  about  leaning  and  instruction.  In 
his  often-cited  article,  Clark  (1989)  agrees  with  Winn, 
and  states  that  researchers  should  conduct  planned 
series  of  studies,  selecting  methods  based  on  sound 
literature  reviews.     His  recommendation  that  we  conduct 
prescriptive  studies  to  answer  why  instructional  design 
methods  work  can  especially  be  followed  by  evaluatore 
using  alternate  research  methods. 

Results  of  Evaluations  of  the  Overall  Effectiveness 
of  Interactive  Technology-Based  Instructional  Programs 

■QfiUEPUter-based.  Instructing 

Recently  several  researchers  have  reported  the 
results  of  meta-analyses  of  general  evaluations  of  the 
effectiveness  of  various  types  of  interactive 
technology-based  programs.     Evaluations  of  computer- 
based  instruction  (CBI)  and  interactive  video  will  be 
presented.     Although  Ambrose  (1991)  has  presented  a 
literature  review  regarding  the  potential  of 
hypermedia,   there  has  not  to  date  been  a  meta-analysis 
of  research  stuaies  indicating  the  effects  of  these 
newer  multimedia  systems  on  learning.     It  is  hoped  that 
such  meta-analyses  may  be  conducted  on  these 
technologies  in  the  future. 

Several  researchers  have  conducted  meta-ana] yse« 
to  etudy  the  overall  effects  of  CA1  on  student 
learning.     For  example,  Kulik,  Kulik  and  Cohen  (1980) 
reviewed  59  evaluations  of  computer-based  college 
teaching.     They  found  that  college  students  who'  learned 
using  computer-based  instruction  (CBI)  generally 
performed  better  on  their  exams  than  students  who 
learned  using  traditional  instruction,  although  the 
differences  were  not  great.     For  example,   they  reported 
that  the  average  exam  score  in  CBI  classes  was  60.6 
percent,  while  the  average  score  in  traditional  classes 
was  57.6.     While  only  eleven  of  the  studies  thev 
reviewed  reported  attitude  data,   thes<    researchers  also 
reported  that  students  who  learned  using  CBI  had  a 
slightly  more  positive  attitude  toward  learning  using 
computers,  and  toward  the  subject  matter.     The  most 
significant  finding  in  this  meta-analysis  was  that  in 
the  eight  studies  which  investigated  effects  of  CBI  on 
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instructional  time,  students  learned  more  quickly  using 
computers  than  in  conventional  classes.     This  finding 
of  this  study  has  often  been  cited  as  one  powerful 
reason  to  use  CBI  in  college  clabses. 

Kulik  and  several  other  researchers  (Kulik, 
Bangert,  &  Williams,   1983)  also  conducted  a  meta- 
analysis on  the  effects  of  computer-based  teaching  on 
learning  of  secondary  school  students.     In  this  study 
they  reviewed  51  evaluations  that  compared  the  final 
examination  scores  of  students  who  had  learned  using 
CBI  with  scores  of  students  who  had  learnea  using 
conventional  methods.     The  results  of  this  study 
indicated  that  learning  using  computers  may  be  even 
more  effective  for  younger  students  than  for  the  older 
students  described  earlier.     Again,   students  in  CBI 
classes  performed  better.     The  average  effect  size  in 
the  CBI  classes  was  .32  standard  deviations  higher  than 
in  conventional  classes.     Another  way  to  describe  this 
difference  would  be  that  students  in  the  CBI  classes 
performed  at  the  83rd  percentile,  while  those  in 
conventional  classes  performed  at  the  50th  percentile. 

In  this  meta-analysis ,  the  researchers  also 
investigated  the  effects  of  CBI  on  student  attitudes. 
There  was  a  small  significantly  positive  effect  of  CBI 
on  attitudes  toward  subject  matter,  computers  and 
instruction.     Only  two  of  the  studies  they  reviewed 
investigated  instructional  time,  and  in  both  students 
learned  more  quickly  using  computers.     Thus,  recent 
studies  have  indicated  the  effectiveness  of  CAI  and  CBI 
on  student  learning. 

Interactive  lisi&Q. 

Savenye   (1990)  presented  findings  of  general 
reviews  of  evaluations  of  the  effectiveness  of 
interactive  video  as  well  as  specific  types  of 
multimedia  studies  which  have  been  conducted.  To 
summarise  her  results,   this  researcher  found  that  in 
the  evaluations  (Bosco,   1986;  DeBloois,   1988;  Slee, 
1989)   interactive  video  generally  helped  students  learn 
better  than  they  did  through  traditional  instruction. 
She  cautioned,  however,   as  did  Bosco,   that  when  studies 
used  statistical  analyses  differences  tended  to  be 
smaller.       In  addition,   this  researcher  reported  that 
learners  usually  have  positive  attitudes  towards 
learning  through  interactive  technologies.     As  in  the 
studios  on  CAI,   researchers  often  found  that  learners 
learn  faster  using  interactive  video. 

McNeil  and  Nelson  (1991)  conducted  a  meta-analysis 
of  studies  which  evaluated  cognitive  achievement  from 
interactive  video  instruction.     These  researchers  used 
criteria  including  presence  of  learning  measures,  use 
of  experimental  or  quasi-experimental  design,  and  sound 
methodology  to  select  63  studies  from  an  initial  list 
of  367.    *0ne  strength  of  their  meta-analysis  is  that 
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many  studies  had  not  been  published,  thus  avoiding  the 
bias  toward  significance  of  published  studies  noted  by 
some  authors. 

Similar  to  Kulik,  et  al.'s  results  of  meta- 
analyses on  CBI,  McNeil  and  Nelson  found  an  overall 
positive  effect  size  (.530,  corrected  for  outliers) 
showing  that  interactive  video  is  effective  for 
instruction . 

These  researchers  conducted  several  types  of 
analyses  in  an  admirable  effort  to  isolate 
instructional  design  factors  which  contribute  to  the 
effect  size .     These  analyses  revealed  that  the  effect 
sizes  were  homogenous,    Mbut  the  selected  independent 
variables  did  not  explain  the  achievement  effect,"  (p. 
5).     They  did,  however,   note  that  there  were  some 
significant  teacher  effects  indicating  that  interactive 
video  was  somewhat  more  effective  when  used  in  groups 
rather  than  individually.     The  authors  remind  us  of  the 
important  role  of  the  teacher*  in  interactive 
instruction.     In  addition,   similar  to  results  noted  by 
Hannafin  (1985),   as  well  as  Steinberg  (1989)  for  some 
types  of  learners,  program  control  appeared  to  be  more 
effective  than  learner  control . 

These  researchers  explained  their  results  by 
noting  that  interactive  video  instruction  consists  of  a 
complex  set  of  interrelated  factors.     Reeves  (1936) 
concurs.     It  will  be  a  continuing  challenge  to 
researchers  studying  interactive  instruction  to  isolate 
factors  crucial  to  the  success  of  innovative 
technologies . 


Planning  Formative  Evaluations  of 
Interactive  Technology-Based  Instruction 


The  following  sections  of  this  paper  will  present 
an  overview  of  planning  and  conducting  formative 
evaluations  of  interactive  instructional  programs.  As 
noted  earlier,   it  is  assumed  that  formative  evaluation 
is  an  on-going  process,  with  activities  conducted 
throughout  all  phases  of  design  and  development. 


Begin  Early 

One  key  to  conducting  cost-ef f ecfcive  and  useful 
formative  evaluations  is  to  begin  planning  early, 
ideally  from  project  inception.     By  beginning  early, 
the  goal  of  the  formative  evaluation  is  determined 
early  as  well.     The  subsequent  processes  and  methods 
can  be  carefully  selected  and  planned  to  collect  the 
most  important  information.     Stakeholders,  managers, 
reviewers,   instructors  and  learners  can  be  identifed 
early,   thereby  limiting  delays  during  development. 
Similarly,  members  of  the  development  team  who  will 
assist  in  data  collection  can  be  identified,  enlisted 
ana  briefed  early.     Early  planning,   in  fact,  can  enable 
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developers  to  collect  data  that  would  be  impossible  to 
collect  if  not  identified  early,  because  the  systems 
(such  as  computer  programs)  to  collect  these  data  might 
not  be  developed  when  needed,  or  at  all. 

Determine  Main  Evaluation  G.oal 

It  is  most  useful  for  communication  and  efficiency 
purposes  for  one  clear  goal  to  be  determined  for  the 
formative  evaluation.     Although  developers  may  want  to 
investigate  many  questions,  the  evaluation  goal  is 
usually  some  variation  of  how  effective  the  interactive 
instruction  is  with  learners  and  instructors .  The 
80/20  rule  applies  as  much  when  conducting  evaluations 
as  it  does  in  most  other  activities,  that  is,  80%  of 
the  benefits  are  derived  from  20%  of  the  effort.  As 
development  progresses  keeping  the  evaluation  goal  in 
mind  will  yield  maximum  results  and  avoid  team  members 
wasting  time  on  less  important  details.     Maintaining  a 
focus  on  one  clear  evaluation  goal  thus  enables 
developers  to  keep  a  view  of  the  forest,   rather  than 
getting  lost  in  the  trees. 

Determine  Manor  Evaluation  Questions  and  JSllbr  quest  ions 

The  major  evaluation  questions  are  derived  from 
the  evaluation  goal.     In  formative  evaluations  of 
interactive  instruction,  as  in  evaluations  of 
instruction  using  other  media,  there  are  typically, 
three  major  evaluation  questions: 


1)  How  well  does  the  instruction  help  the 
students  learn  ( an  achievement  question ) ? 


2)  How  do  the  learners,   instructors,  and 
other  users  or  constituents  feel  about  the 
instruction  (an  attitude  question)? 


3 )  How  is  the  instruct  ion  implemented  ( a 
"use "  question) ? 


(Higgins  &  Sullivan,   1982;  Morris  &  Fitzgibbons,  1979; 
Sullivan  &  Higgins,  1980). 

To  answer  each  question,  evaluators  and  developers 
determine  data  to  be  collected,   select  data  collection 
methods ,  develop  instruments  and  procedures ,  and 
determine  how  data  will  be  analyzed.     One  way  to  plan 
the  evaluations  to  both  keep  the  focus  clear  and  make 
procedures  most  efficient  is  to  develop  a  matrix  to 
guide  the  evaluation  and  development  team.     Under  each 
major  evaluation  question  can  be  listed  the  related 
subquestions.     Beside  each  question,   as  headings  across 
the  matrix  would  be  "data  sources"  (instructors, 
learners,  administrators,  expert  reviewers,   etc.  )  , 
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"data  collection  methods"  (measures  of  initial 
learning,  measures  of  learning  transfer,  attitude 
surveys,   interviews,  observations,  etc.),  and 
"instruments"   (pretests  and  posttests  of  achievement, 
instructor  questionnaires ,  observational  checklists , 
etc. )   (cf .   Savenye,   1986a) . 

Interactive  technologies  both  allow  for,  and  call 
for,   different  subquestions  related  to  the  three  major 
types  of  evaluation  questons.     They  also  call  for 
expanded  views  of  what  data  can  be  collected  and 
analyzed  and  how  these  data  can  be  used.  Developers 
and  evaluators  of  interactive  programs  have  a 
responsibility  to  add  new  methods  to  their  evaluation 
"toolkits",  to  maintain  an  open  view  with  regard  to 
questions  which  need  to  be  answered,  as  well  as  to 
report  the  results  of  their  evaluations  to  benefit 
their  colleagues  who  develop  interactive  technology- 
based  instruction  in  all  settings.     The  latter 
responsibility,   in  particular,  has  been  noted  by  many 
authors  (cf.  Clark,   1989;  Patterson  &  Bloch,  1987; 
Reigeluth,   1989;  Winn,  1989). 

Alternate  methods  of  conducting  evaluations 
are  most  useful,    in  fact  may  be  critical,   in  answering 
the  third  major  type  of  evaluation  question  -  how  is 
the  interactive  instruction  implemented  or  used. 
Flexible,  open  views  with  regard  to  "what  is  really 
happening"  when  innovative  approaches  and  technologies 
are  used  can  result  in  finding  that  a  critical 
component  of  instructor  training,   for  example,  had  been 
left  out  jf  the  initial  design,  or  that  learners  are 
using  the  technology  in  ways  developers  never 
anticipated;   in  fact,   they  may  be  using  it  in  better 
ways.     This  can  yield  what  Newman  (1989)  calls  answers 
to  how  the  learning  environment  is  affecting  the 
instructional  technology.     Newman  elucidates:   "How  a 
new  piece  of  educational  technology  gets  used  in  a 
particular  environment  cannot  always  be  anticipated 
ahead  of  time.     It  can  be  argued  that  what  the 
environment  does  with  the  technology  provides  critical 
information  to  guide  design  process"   (p.    1).     He  adds, 
"It  is  seldom  the  case  that  the  technology  can  be 
inserted  into  a  classroom  without  changing  other 
aspects  of  the  environment,"   (p.  3),  a  fact  often  noted 
by  instructional  systems  designers . 

When  such  questions  are  not  brought  up  and 
investigated,   an  instructional  innovation  can  fail,  as 
those  who  developed  "programmed  learning"  in  the 
sixties,  or  who  have  implemented  educational 
technologies  in  other  cultural  settings  without  getting 
participant  "buy- in"  have  learned.     In  other  words, 
selecting  evaluation  methods  with  a  critical  eye  toward 
the  realities  of  what  can  be  happening  when  we  use  new 
technologies  is  callfad  for. 

In  addition,  with  the  prospect  of  continued  lack 
of  support  for  "ba&ic  research"  in  educational 
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technology,  developers  can  contribute  to  the  knowledge 
base  in  our  field  by  conducting  "applied  research"  in 
the  form  of  rigorous  high-quality  formative  evaluations 
and  publishing  their  methods  and  results.  Alternate 
research  methods  can  be  used  carefully  to  answer  "open 
questions"  related  to  implementation  of  technology, 
such  as  how  youngsters  make  decisions  as  they  proceed 
through  a  simulation,  or  how  teachers  use  an 
interactive  videodisc  program  with  whole  classes- 
Results  reported  by  an  evaluator  in  one  study  can  yield 
instructional  design  and  implementation  guidelines  that 
developers  can  use  and  test.     No  less  important  to  the 
continued  improvement  in  our  knowledge  is  that 
researchers  can  use  results  of  "naturalistic  methods" 
to  formulate  questions  and  isolate  factors  which  can 
subsequently  be  investigated  using  experimental  method, 
yielding  causal  interpretations . 

The  following  section  of  this  paper  will  present  a 
discussion  of  multiple  methods  for  conducting  formative 
evaluations  of  interactive  instruction,  with  particular 
attention  to  selecting  appropriate  methods- 
Data  Collection  and  Analysis  Methods 

While  the  goal  of  formative  evaluation  is  to 
improve  the  learning  effectiveness  of  the  programs,  the 
choice  of  methods  for  conducting  evaluations  is  not 
clear-cut.     As  recommended  by  Jacob  (1987)  in  her 
review  of  qualitative  research  traditions,  methods 
should  be  chosen  based  on  the  research  questions  to  be 
answered.     In  the  case  of  evaluation,  where  resources 
are  limited  and  the  value  of  the  process  is  not  always 
clear  to  constituents,   selecting  methods  should  be 
driven  by  evaluation  questions. 

In  addition,   it  is  important  that  developers  and 
evaluators  contribute  to  our  knowledge  of  effects  of 
instructional  design  factors.  As  noted  by  many 
researchers  (McNeil  &  Nelson,   1991;  Reeves,  1986), 
instruction  based  on  interactive  technologies  relies  on 
many  individual  factors  for  its  success,   and  each 
program  is  often  unique  in  its  approach,  use  of  media, 
etc.     The  challenge  to  determine  what  factors  make  a 
difference  in  learning  is  great . 

In  the  discussion  below  will  be  interwoven  the 
utility  of  various  alternate  research  methods  with 
traditional  methods.     The  methods  will  be  presented 
with  relation  to  major  areas  of  evaluation  types. 

Instructional  Design  Reviews 

Most  instructional  design  models  include  the 
recommendation  that  draft  versions  of  instructional 
materials  be  reviewed  periodically  during  development. 
It  is  particularly  important  that  aspects  of 
interactive  programs  be  reviewed  at  many  stages  during 
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development.     Geis  (1987)  recommends  that  materials 
might  be  reviewed  by  subject  matter  experts, 
instructional  designers,   technical  designers  such  as 
graphic  artists ,  instructors ,   individuals  who  have 
special  knowledge  of  the  target  audience,  influential 
community  leaders,   project  sponsors,  previous  students 
and  project  editors-     In  a  large-scale  school  science 
project,   for  example,   initial  objectives  might  be  sent 
with  brief  descriptions  of  video  and  computer 
treatments  for  lesson  segments  to  scientists, 
instructional  designers  and  teachers  using  other 
materials  developed  by  the  organization.  Subsequent 
reviews  might  elicit  responses  to  depictions  of 
computer  menus,  descriptions  of  branching  options  and 
simulation  and  game  segments,  as  well  a&  video 
storyboards . 

Whi le  use  of  drafts  of  print  materials ,  scripts 
and  storyboards  for  reviews  is  traditional  in  formative 
evaluation  reviews,  it  should  be  noted  that  computer 
programs ,   interactive  video  lessons ,   and  interactive 
multimedia  presentations  are  often  too  complex  for  many 
reviewers  to  evaluate  in  print  form.     Many  evaluators, 
therefore,  submit  prototype  versions  of  aspects  01 
lessons,   such  as  crucial  menus,  operational  draft 
segments  of  simulations,  or  selected  lessons  to 
reviewers.     Reviews  are  typically  solicited  early 
during  formative  evaluation  activities  to  answer 
format,   style,   and  content  questions,  and  reviews 
continue  on  an  on-going  basis. 

Determining  Ls^mLng  Achievement 

Paper-basgd  Tests.     Traditional  measures  of 
achievement  are  still  appropriate  for  use  in 
determining  how  well  learners  perform  after  completing 
interactive  lessons.     Such  measures  are  usually  forms 
of  paper~and~pencil  tests.     What  is  critical  is  that 
the  test  items  match  the  learning  objectives  developed 
during  design  (Dick  &  Carey,   1990;  Higgins  &  Rice, 
1991;   Sullivan  &  Higgins,   1983).     Without  such  a  match 
the  test  is  often  not  useful,  and,  unfortunately  this 
can  often  be  the  case  in  evaluating  interactive 
programs  in  which  developers  let  technical   "bells  and 
whistles"  drive  the  design  process.     The  decision  to 
use  paper-and-pencil  tests  is  often  made  based  on 
practical,  considerations,  3uch  as  the  fact  that  there 
may  not  be  enough  delivery  systems  for  each  student  in 
a  class,  or  that  tests  must  be  taken  after  students 
have  left  the  training  setting,  or  due  to  time 
limitations  in  accessing  equipment.     There  is  a  danger, 
however,   in  using  paper-and-pencil  tests  when  learners 
received  their  practice  in  lessons  through  the  computer 
or  other  technology.     The  "conditions"  of  the 
performance  in  the  test  may  no  longer  match  that  of  the 
objectives.     Even  a  difference  such  as  having  computer 
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graphics  with  text  in  practice  activities  with  text 
only  in  the  paper-based  test  can  invalidate  the  test 
items* .     Technology-based  tests  to  match  the  lesson 
objectives,  format,  and  practice  c:re  thus  somewhat 
preferable,  unless  the  paper -and-pencil  tests  and 
technology-based  practice  are  carefully  matched. 

Iffighna LaggzJaftS-Bd  I98t8,  Computer-based 
achievement  tests  offer  other  advantages  to  paper-based 
tests.     A  test-item  bank  can  be  developed  to  allow 
administering  multiple  forms  of  the  test.  Data 
collection  can  be  greatly  simplified,   in  that  computer 
programs  can  be  written  to  transfer  performance  data 
directly  to  files  for  data  analysis.     (Of  course,  use 
of  optical ly-scannable  answer  sheets  with  paper  tests 
also  increases  efficiency).     Adaptive  tests  might  be 
developed  that  present  specified  items  based  on 
performance,  and,  once  a  student  has  begun  to  fail 
items  based  on  a  knowledge  or  skill  hierarchy,  for 
example,  save  testing  time  by  not  administering  more 
items  for  advanced  skills. 


Performance ,     A  critical  issue  in  evaluating  learning 
is  often  how  well  students  perform  in  their  real-world 
settings.     Although  most  instructional  developers  have 
traditionally  recommended  evaluating  on-the-job 
performance,   the  efficiency  of  using  less-realistic 
measures  often  ensures  that  paper-and-pencil  or 
computer-based  tests  are  used.     Observations  of  learner 
performance  in  any  work  or  life  setting  can,  however, 
be  conducted  using  methods  adapted  from  ethnographic 
research . 

Should  evaluators  decide  to  conduct  observations, 
several  decisions  must  be  made.     The  team  should 
determine  who  will  conduct  the  observations,  how  the 
observers  will  be  trained  to  ensure  consistency,  on 
what  performances  they  will  collect  data,  how 
observations  will  be  recorded,   how  inter-observer 
reliability  will  be  determined,  how  the  data  will  be 
analyzed  and  how  the  results  will  be  reported.  For 
example,   if  the  learned  task  is  primarily  procedural, 
it  may  be  a  simple  matter  to  develop  a  checklist  for 
recording  how  closely  a  student  follows  the  required 
procedural  steps  in  an  assessment  situation.  In 
contrast,   if  the  learned  task  was  a  more  "fuzzy"  type 
of  skill,   such  as  how  to  conduct  an  employment 
interview,   the  observational  procedures ,   checklists , 
etc.,  would  be  more  complex,  and  reliability  of 
observations  could  be  a  trickier  issue,   due  to 
subjectivity  of  what  observers  might  be  recording. 
Conducting  observations  of  behaviors  will  be  discussed 
further  in  a  section  concerned  with  evaluating 
implementation  of  interactive  systems  in  real-world 
learning  environments. 

It  might  be  noted  that  when  it  is  not  practical  to 
observe  student  performance  on  the  job  or  out  of 
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school,   it  may  still  be  practical  to  conduct 
observations  of  students  in  "classroom"  settings  who 
are  engaged  in  formal  role-plays  or  simulations  of  the 
skills  they  learned  through  interactive  instruction. 
The  considerations  described  above  would  also  be 
relevant  in  evaluating  learning  through  such 
simulations . 

Products/Portfolios,     As  noted  by  Linn,   Baker  and 
Dunbar  (1991),  there  is  increased  concern  among 
educators  that  traditional  assessment  methods 
shortchange  evaluation  of  complex  performance-based 
learning.     In  school,  settings,   for  example  in  programs 
to  determine  which  students  to  include  in  gifted  and 
talented  programs,   it  is  becoming  common  to  include 
portfolios  of  student  writing  and  other  samples  of  the 
products  of  students'  work.     Interactive  technology- 
based  systems  are  often  developed  specifically  to  teach 
complex  sets  of  behaviors  and  problem-solving  skills 
through  simulations.     It  is  to  be  expected  that 
instructional  developers  of  interactive  learning 
systems  would  collect  products  of  student  work  to 
directly  measure  achievement  of  complex  objectives- 

For  example,  a  student  who  learned  to  repair 
equipment  by  experiencing  computer-and-videodisc 
simulations  could  be  expected  to  demonstrate  learning 
achievement  by  repairing  an  actual  piece  of 
malfunctioning  machinery.     The  repaired  equipment  would 
thus  be  a  product.     Here  again,  as  recommended  by  Dick 
and  Carey  (1990),  Sullivan  and  Higgins  (1983),   and  most 
other  instructional  developers ,  an  evaluation  checklist 
would  be  developed  to  determine  mastery  of  the  skill  as 
demonstrated  by  the  quality  of  the  product. 

Similarly,   if  a  student  learned  to  create  an  art 
piece  by  participating  in  a  videodisc-based  interactive 
lesson  about  a  particular  type  of  art,   the  evaluation 
would  logically  involve  determining  the  quality  of  the 
student's  creation,  according  to  criteria  established 
in  the  lesson. 

One  caution  that  applies  in  all  type^  of 
evaluations  of  student  products  and  portfolios  relates 
to  the  alignment  between  practice  and  assessment 
activities .     Developers  and  evaluators  cannot  expect 
learners  to  move  directly  from  doing  practice  in  a 
technology -based  simulation  to  performing  the  skill  in 
the  real  world  setting.     As  noted  earlier,  the 
conditions  of  the  practice  and  assessment  in  this 
situation  would  not  match.     Developers  would  do  well  to 
ensure  that  learners  engaged  in  learning  using  their 
instructional  system  receive  some  type  of  practice  on 
the  actual  equipment  or  in  the  real-world  setting,  or 
producing  the  real  product,  before  they  are  tested  in 
the  latter  situations . 

lime.  Measures,     For  some  types  of  learning, 
mastery  is  measured  by  the  quality  or  frequency  of 
student  performance  within  given  time  parameters.  This 
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would  be  the  case,   for  example,   for  keyboarding  skills 
in  which  mastery  is  demonstrated  both  by  accuracy  and 
speed.     Interactive  technology-based  systems  can  easily 
record  how  quickly  or  frequently  learners  perform.  In 
addition,   it  has  often  been  noted  that  the  "claim  to 
fame"  for  technology  is  that  it  helps  learners  progress 
to  mastery  levels  more  quickly  than  through  traditional 
instruction.     Many  evaluations  of  interactive 
instruction  therefore  include  measures  of  time  to 
mastery.     It  is  likely  that  evaluations  of  evolving 
systems  will  continue  to  include  collecting  time  data. 

Self -Evaluation-     In  some  instances ,  particularly 
in  adult  or  recreational  learning  settings,  collecting 
data  regarding  learners'  perceptions  of  their  own 
achievement  of  skills  is  desirable.     Such  data  can  be 
collected  using  straightforward  questions  on  survey 
instruments,   such  as  "How  would  you  rate  your  skill  in 

  now?"     Such  self-report  data  is  often  biased, 

however,  and  so  it  is  usually  more  useful  to  collect 
data  which  directly  measures  student  learning. 
However,  at  times,  developers  may  also  be  concerned 
with  learners'  perceptions  of  their  learning,  perhaps 
for  political  reasons,  and  these  may  be  useful 
depending  on  the  evaluation  questions. 

Interviews.     Although  not  typically  used  to 
collect  achievement  data ;   there  are  a  few  instances  in 
which  interviews  might  be  useful.     Interviews  may  be 
conducted  to  collect  self -evaluation  data.  In 
addition,  with  very  young  students  or  those  who  are  not 
literate  interviews  may  rrally  be  oral  tests  conducted 
to  measure  learning  achievement. 

Occasionnally ,  especially  in  training  settings, 
, interviews  are  conducted  with  managers  to,  determine  how 
well  they  believe  employees  learned  the  skills 
practiced  through  interactive  instruction,  and  how  well 
managers  believe  employees  are  now  performing  in  their 
jobs.     Collecting  these  data  sometimes  has  the  side 
benefit  of  contributing  to  managers'   "buy- in"  of  the 
interactive  training,  as  they  reflect  on  what  their 
employees  learned  through  the  training. 

Documentary  I^ta-     In  some  settings,  evaluators 
of  interactive  technologies  will  secure  access  to 
data  already  existing  in  the  organization.  In 
educational  settings  these  data  might  be  end-of -course 
grades-     For  example,   the  final  grades  of  college 
students'  who  completed  a  course  delivered  via 
interactive  video  might  be  compared  with  those  of 
students  who  completed  the  course  in  a  traditional 
manner.     In  schools  districts,  evaluators  may  secure 
access  to  student  performance  on  yearly  standardized 
tests.     In  both  these  cases,   these  data  would  be  more 
relevant  for  whole  courses  which  used  interactive 
technology  than  for  those  courses  which  employed 
technology  in  a  supplementary  and  limited  manner. 

In  training  settings,  evaluators  might  review 
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industrial  documentary  data  regarding  increased 
production,  decreased  loss  due  to  error,  decreased 
reports  of  health  or  safety  violations,  reduced 
customer  complaints ,  increased  efficiency ,  etc . 

Issues  Related  jta  Transfer.    A  particularly  sticky 
issue  in  education  is  hov;  well  learners  can  perform  in 
real-world  settings  the  skills  they  mastered  in 
artificial  settings  such  as  classrooms .     Many  advocates 
of  multimedia  argue  that  interactive  instructional 
programs  can  closely  simulate  the  real-world,  sometimes 
calling  such  systems  learning  environments.  For 
example,  one  interactive  video  curriculum  has  been 
developed  to  train  reserve  soldiers  to  repair  and 
maintain  the  M-l  tank  (Savenye,   1986b).     Yet  in  this 
project  military  trainers  noted  that  troubleshooting  a 
firing  system  malfunction  based  on  video  and  audio 
displays,  and  then  selecting  a  decision  such  as 
replacing  a  part  from  icons  on  a  menu  is  not  the  same 
as  actually  performing  these  activities  on  a  tank. 
While  few  evaluations  have  measured  learning  transfer 
to  on-the-job  or  outside-school  tasks,   some  studies 
have  indicated  learning  through  interactive  media  does 
help  students  learn  to  transfer  their  knowledge  to 
other  settings  more  quickly  than  learning  through 
traditional  instruction  (DeBloois,  1988). 

It  will  remain  a  responsibility  of  developers  to 
use  technology  to  build  learning  systems,  especially 
simulations,  that  enhance  learning  transfer,   ana  of 
evaluators  to  creatively  measure  such  transfer. 

Issues  Related  to  Retention.     Regardless  of  how 
learning  is  measured  it  is  advisable  to  administer 
delayed  versions  of  tests  or  other  measures  to 
determine  how  much  learner  retain  of  what  they  have 
learned.     It  is  not  difficult  in  on-going  interactive 
curriculum  materials  to  build  in  periodic  tests  which 
students  might  view  as  "reviews",  but  which  developers 
could  use  to  measure  retention. 

Answering  Other  Types  q£  Learning  Questions 

Interactive  technology-based  instruction  may  be 
used  in  nontradit ional  educational  settings,   such  as  in 
museums  and  parks,  or  even  for  delivery  of  information, 
as  opposed  to  instruction.     In  these  cases  the  learning 
to  be  measured  may  be  quite  different  from  achievement 
of  learning  objectives  and  the  evaluation  questions , 
therefore,  may  differ. 

An  example  of  such  a  situation  was  presented  by 
Hirumi,  Allen  and  Savenye  (1989).     These  authors 
discussed  the  development  and  evaluation  of  an 
interactive  videodisc-based  museum  exhibit  to  introduce 
visitors  to  the  plants  and  animals  of  the  desert.     In  a 
museum  setting  visitors  experience  an  exhibit  in 
groups,  with  only  one  or  two  individuals  actually 
making  choices  on  the  computer.     Visitors  spend  little 
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time  with  an  exhibit,  and  are  unlikely  to  be  willing  to 
take  traditional  tests.     In  this  type  of  setting,  if 
learning  from  the  interactive  exhibit  is  a  concern,  a 
limited  number  of  nonthreatening  learning  achievement 
questions  can  be  asked  of  samples  of  visitors  in  very 
brief  interviews.     If  classes  of  children  visit  the 
museum  it  is  often  possible  to  ask  them  to  complete 
short  activities  which  they  may  perceive  as  fun,  but 
which  actually  measure  learning. 

Technology-based  learning  environments  may  be 
developed  with  broader  goals  than  to  teach  specific 
objectives.     They  may,  for  example,  be  based  on  an 
exploratory  learning  approach  with  the  goal  of 
enhancing  student  motivation  to  prepare  them  to 
participate  in  more  structured  learning  activities 
later*     Such  exploratory  systems  may  include  a 
videodisc  that  shows  students  whatever  aspects  of  a 
setting  they  may  choose,   for  example  selected  parts  of 
a  town  or  archaeological  site,  or  the  flora  and  fauna 
of  natural  surroundings.     These  systems  often  are  based 
on  hypertext,   and  thus  allow  learners  to  branch  in  a 
network  fashion  from  any  bit  of  information  in  the 
database  to  any  other.     Evaluators  investigating 
effects  of  such  exploratory  learning  environments  may, 
depending  on  the  evaluation  questions,  collect 
computerized  data  on  the  pathways  learners  take  through 
information,   and  what  choices  they  make.     If  most 
learners  bypass  some  parts  of  the  information,  for 
example,  or  always  go  through  some  parts,  evaluators 
could  conduct  followup  interviews  to  ask  learners  why 
they  make  the  choices  they  do. 

A  technique  of  using  read-think~alouu  protocols 
could  also  be  used  (Smith  &  Wedman,   1988)  to  analyze 
learner  tracking  and  choices.     Using  this  technique, 
evaluators  could  ask  learners  to  "talk  through"  their 
decisions  as  they  go  through  a  lesson.  Evaluators 
could  observe  ^nd  listen  as  learners  participate,  or 
they  could  audiotape  the  learners  and  analyze  the  tapes 
later.     In  either  case,  the  resulting  verbal  data  must 
be  coded  and  summarized  to  answer  the  evaluation 
questions.     Techniques  of  protocol  analysis  (cf. 
Ericsson  &  Simon,   1984)  should  be  determined  and  tested 
early  in  the  evaluation  process. 

As  described  earlier,  observations  of  actual 
or  simulated  performances  may  be  called  for,  although 
in  these  nontraditional  learning  environments 
performance  is  not  always  as  much  a  concern  as  is 
motivation . 

In  contrast  to  these  nontraditional  educational 
settings,   in  which  performance  is  not  always  critical, 
business  and  industrial  settings  in  which  training  is 
delivered  on-line  or  on-demand,  do  hold  learner 
performance  to  be  of  utmost  concern.     Yet  these 
settings  do  not  always  allow  for  traditional  testing. 
It  is  not  difficult,  however,  to  develop  unobtrusive, 
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objectives-based  performance  measures  that  are  resident 
in  the  computer  system,  and  which  learners  would  not 
object  to.     For  example,   if  an  employee  calls  up  a 
brief  tutorial  while  attempting  to  use  a  software 
feature  which  is  new  to  her,  her  subsequent  performance 
using  the  feature  could  be  measured  and  recorded  by  the 
computer  system.     In  training  settings  issues  of 
confidentiality  of  performance  achievement  may  arise, 
so  evaluators  and  managers  together  might  determine 
whether  and  how  employees  would  be  informed  that  their 
performance  would  be  tracked,  and  how  those  data  would 
be  used. 

As  in  the  earlier  discussion,   in  these  settings, 
data  regarding  learning  time  and  self -evaluation  of 
performance,  as  well  as  documentary  and  interview  lata 
could  be  collected. 

Determining  Atti tudes/Perceptions 

The  traditional  methods  of  collecting  data  on 
the  attitudes  and  perceptions  of  learners  and  other 
stakeholders  have  included  questionnaires  and,  less 
frequently,   interviews.     The  traditional  issues  of 
sampling  and  how  to  compare  results  of  instructional 
methods  continue  to  apply  when  evaluating  interactive 
instruction . 

Questionnaires.     Using  quantitative  methods 
evaluators  may  wish  to  compare  the 

attitudes  toward  a  subject  of  students  who  learned  the 
subject  through  interactive  instruction  with  students 
who  participated  in  a  traditional  course.  This 
approach  was  used  by  Savenye  (1989)  who  found  that 
students  who  participated  in  a  full-year  videdisc-based 
high  school  physical  science  curriculum  generally  held 
more  positive  attitudes  toward  science  and  how  uhey 
learned  science  than  students  who  took  the  course  via 
traditional  instruct  ion .     In  this  evaluation ,  as 
always,  sufficient  numbers  of  students  needed  to 
complete  the  surveys  to  yield  reliable  results. 
Additionally,  care  was  taken  to  include  students  from 
various  types  of  schools  and  communities,  such  as 
urban ,   rural  and  suburban ,   and  from  representative 
geographic  areas  and  cultural  groups  in  the  evaluation. 

If  an  interactive  program  had  as  its  primary  goal 
an  improvement  in  attitudes,  evaluators  are  likely  to 
need  to  collect  preinstruct ional  and  post instructional 
attitude  data.     In  a  related  example,   Savenye,  Davidson 
&  Orr  (1992)  collected  pre  and  post  data,  arid  reported 
that  preservice  teachers *  attitudes  toward  computers 
were  higher,  and  their  anxiety  lower,   after  they  had 
participated  in  an  intensive  computer  applications 
course . 

Questionnaire  items  and  directions  should  be 
clearly-written,   arid  the  questionnaire  should  be  as 
short  as  possible.     Questions  should  be  directly  based 
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on  the  needs  of  the  evaluation.     Evaluators  may  wisia  to 
make  most  items  forced-response,  such  as  Likert-scale 
items,  to  speed  data  analysis.     A  case  can  be  made, 
however,   for  including  a  few  open-ended  questions  in 
every  survey,   to  allow  learners  to  bring  up  issues  not 
anticipated  by  evaluators,  who  may  be  unfamiliar  with 
the  learners'  needs  and  concerns  and  constraints  of 
their  learning  environment . 

Tyiterviewff ,     Attitudes  can  also  be  measured  using 
interviews .     Often  evaluators  supplement  questionnaires 
by  conducting  one-on-one  or  small,   focus-group  type, 
interviews  with  a  small  sample  of  learners  to  ensure 
that  all  relevant  data  were  collected  and  nothing  was 
missed  by  using  a  questionnaire.     When  budget  is 
limited,   interviews  may  be  the  sole  means  of  collected 
attitude  data,  primarily  to  verify  and  explain 
achievement  results.     For  example,  Nielsen  (1990) 
incorporated  interviews  into  his  experimental 
study  investigating  achievement  effects  of 
informational  feedback  and  second  attempt  in  computer- 
aided  learning.     Nielsen  fcund  that  some  of  his 
learners,  who  not  coincidentally  were  highly  motivated 
Air  B'orce  cadets,  who  received  no  feedback  determined 
that  their  performance  depended  more  on  their  own  hard 
work  and  they  took  longer  to  study  the  lesson,  while 
the  cadets  who  received  the  extensive  informational 
feedback  soon  figured  out  they  would  receive  the 
answers  anyway,  and  so  spent  less  time  on  the  practice 
items . 

Usually  it  is  desirable  when  conducting 
interviews,  particularly  when  several  evaluators  will 
be  interviewing  learners,  for  a  set  of  structured 
questions  to  be  developed.     Otherwise  ideosyncrat ic 
data  may  accidentally  be  collected  from  each  learner, 
and  data  will  not  be  comparable.     In  addition,   it  is 
usually  useful  for  interviewers  to  be  given  the  freedom 
to  probe  further  as  the  interviews  progress, 
particularly  when  the  evaluation  involves  a  completely 
new  interactive  system,  which  may  be  causing  many  types 
of  changes  in  the  instructional  setting. 

Learner  Notes»     Ih  traditional  field  tests, 
evaluators  often  collect  attitude  data  by  allowing 
learners  to  write  comments  on  their  materials.  In 
interactive  systems,  a  computer  program  can  be  written 
to  easily  allow  learners  to  write  notes  and  comments  to 
developers . 

Q£h£E  Types  a£  Collection  Methods,  As 


described  earlier,   it  may  be  desirable  to  observe 
learners  and  collect  incidental  attitude  data,  provided 
observers  have  agreed  what  type  of  data  they  will 
record.     Additional  data  can  also  come  up  in  interviews 
or  through  collecting  documentary  data.     One  example  of 
such  data  was  the  observation  by  teachers  and 
evaluators  in  many  schools  which  used  a  videodisc-based 
science  curriculum  that  many  more  students  were  coming 
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into  the  science  classrooms  to  "play"  with  the  science 
lessorb  during  their  free  time  than  ever  occurred  when 
traditional  science  lessons  were  being  used. 

An  exemplary  study  using  alternate  research 
methods  to  determine  teacher  perceptions  of  competency- 
based  testing  serves  as  an  example  of  what  can  be  done 
to  measure  attitudes  and  perceptions.     Higgins  and  Rice 
(1991)  conducted  a  three-phase  study.     They  initially 
conducted  relatively  unstructured  interviews  with  six 
teachers  regarding  the  methods  they  used  to  assess 
their  students,  and  in  what  situations  they  used  the 
techniques .     From  these  interviews  the  researchers 
constructed  a  taxonomy  of  assessments  methods.  These 
researchers  the  employed  trained  observers  to  collect 
data  during  ten  hours  of  classroom  observations 
regarding  how  teachers  measured  their  students. 
Subsequent  interviews  were  conducted  to  ask  teachers 
their  perceptions  of  how  they  were  using  assessments 
during  their  classes,  and  to  have  teachers  rank  their 
perceptions  of  the  utility  and  similarity  of  the  types 
of  assessments  the  tea  hers  had  described.  The 
interview  and  observation  data  were  coded  and 
summarized .     The  rankings  from  the  teacher  interviews 
were  used  to  perform  mult imensional  scaling,  which 
yielded  a  two-dimensional  representation  of  the 
teachers"  perceptions.     Similar  techniques  could  be 
adapted  by  evaluators  to  answer  questions  related  to 
instructor  and  learner  perceptions  of  their  technology- 
based  lessons,  or  their  attitudes  toward  content  and 
skills  learned. 


Evaluating  Use/Implementation 

It  is  in  answering  questions  related  to  how  the 
interactive  instruction  is  being  used  in  the  various 
learning  settings  that  evaluators  can  most  profitably 
use  alternate  methodologies.     The  most  efficient,  and 
therefore ,   first  methods  to  use  to  answer 
implementation  questions  are  still  questionnaires  and 
interviews.     However,   sometimes  the  question  when  using 
a  truly  new  technology  is  often,   "What  is  really 
happening  here,"  as  opposed  to  what  developers  may  plan 
to  or  hope  to  happen.     Here  we  especially  need  answers 
that  ring  true,  ana  here  we  sometimes  do  not  know  the 
right  questions  to  ask.     Using  an  anthropological 
approach,  evaluators  can  go  into  their  learning 
settings  with  an  open  mind. 

Participant  Observation.     Participant  observation 
is  a  technique  derived  from  ethnographic  studies.  It 
involves  intensive  observation  of  partipants  in  a 
setting.     Anthropologists  may  spend  years  "in  the 
field"  becoming  in  a  sense  members  of  a  community, 
therefore  participants,  while  they  observe  and  record 
the  patterns  and  interactions  of  people  in  that  community. 

Evaluators  often  cannot,  nor  do  they  need  to, 
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spend  as  much  time  in  their  instructional  settings,  as 
do  anthropologists,  yet  the  activity  is  extremely 
labor-intensive,  and  data  collection  is  usually  limited 
to  those  data  which  will  answer  the  questions  at  hand. 
Still,  evaluators  would  do  well  to  remember  that 
although  they  do  not  spend  years  observing  the 
particular  instructional  community,  they  do  quickly 
become  participants .     Their  presence  may  influence 
results,  and  their  experience  may  bias  what  they 
observe  and  record-     In  subsequent  reports,  therefore, 
this  subjectivity  can  simply  be  honestly  acknowledged. 

Methods  of  collecting  observational  data  may 
include  writing  down  all  that  occurs,  or  recording 
using  a  limited  checklist  of  behaviors.       Observers  can 
watch  and  write  as  they  go,  or  data  can  be  collected 
using  videotapes  or  audiotapes.     As  mentioned  earlier, 
analyzing  qualitative  data  is  problematic.  Every 
behavior  that  instructors  and  students  engage  in  could 
potentially  be  recorded  and  analyzed,  but  this  can  be 
costly  in  money  and  hours,   and  would  most  likely  be 
useless  for  evaluation  purposes.     Evaluators  should 
determine  in  advance  what  they  need  to  find  out. 

For  example,  Savenye  &  Strand  (1989)   in  the 
initial  pilot  test  and  Savenye  (1989)   in  the  subsequent 
larger  field  test  of  the  science  videodisc  curriculum 
described  earlier  determined  that  what  was  of  most 
concern  during  implementation  was  how  teachers  used  the 
curriculum.     Among  other  questions,  developers  were 
interested  in  how  much  teachers  followed  the  teachers" 
guide,   the  types  of  questions  they  asked  students  when 
the  system  paused  for  class  discussion,   and  what 
teachers  added  to  or  didn't  use  from  the  curriculum.  A 
careful  sample  of  classroom  lessons  was  videotaped  and 
the  data  coded.     For  example,   teacher  questions  were 
coded  according  to  a  taxonomy  based  on  Bloom*' s  (1934), 
and  results  indicated  that  teachers  typically  used  the 
system  pauses  to  ask  recall-level,   rather  than  higher- 
level  questions.     Analysis  of  the  coded  behaviors  for 
what  teachers  added  indicated  that  most  of  the  teachers 
in  the  sample  added  examples  to  the  lessons  that  would 
add  relevance  to  their  own  learners,   and  that  almost 
all  of  the  teachers  added  reviews  of  the  previous 
lessons  to  the  beginning  of  the  new  lesson-  Sonu. 
teachers  seemed  to  feel  they  needed  to  continue  to 
lecture  their  classes,  therefore  they  duplicated  the 
content  presented  in  the  interactive  lessons. 
Developers  used  the  results  of  these  evaluations  to 
make  changes  in  the  curriculum  and  in  the  teacher 
training  that  accompanied  the  curriculum.     Of  interest 
in  this  evaluation  was  a  comparison  of  these  varied 
teacher  behaviors  with  the  student  achievement  results. 
Borich  (1989)  found  that  learning  achievement  among 
students  who  u^ed  the  interactive  videodisc  curriculum 
was  significantly  higher  than  among  control  students . 
Therefore  teachers  had  a  great  decree  of  freedom  in 


using  the  curriculum  and  the  students  still  learned 
well . 

If  the  student  use  of  interactive  lessons  was  the 
major  concern,  evaluators  might  videotape  samples  of 
students  using  an  interactive  lesson  in  cooperative 
groups,  and  code  student  statements  and  behaviors,  as 
did  Schmidt  (1992) . 

Reporting  Results 

How  the  results  of  formative  evaluations  are 
reported  depends  on  how  the  results  are  to  be  use.  For 
example,   if  the  report  is  for  a  funding  source,   or  to 
ensure  continuing  support  for  a  large  project ,  the 
report  might  be  quite  formal  and  detailed.  In 
contrast,   if  the  results  of  the  formative  evaluations 
are  for  immediate  use  by  the  development  team  only,  the 
reports  may  consist  of  informal  summaries,  memos  and 
briefings . 

The  primary  rule  in  reporting  is  to  keep  it 
simple.     Long  evaluation  reports  may  not  be  read  by 
those  who  most  need  them. 

The  organization  of  the  report  may  best  be 
accomplished  by  using  the  evaluation  questions  as 
headings  and  answering  each  question  in  the  sequence 
the  audience  most  likely  would  desire. 

At  a  minimum  the  report  should  usually  include 
sections  on  learning  achievement,  attitudes,  and 
vide/implementation.     With  regard  to  achievement,  at 
least  the  major  mean  scores  should  be  reported,  with  a 
summary  table  typically  included.     Results  of  any 
statistical  comparisons  may  be  reported.     Finally  other 
learning  results  or  anecdoctal  data  related  to 
performance,   such  as  the  results  of  interviews, 
observations,  or  analysis  of  products  or  documentary 
data  should  be  reported  here  (cf.  Dick  &  Carey,  1990). 

When  reporting  attitudes,   the  primary  findings 
related  to  the  evaluation  questions  can  be  described. 
It  may  be  desirable  to  summarize  the  results  of  survey 
items  on  a  copy  of  the  survey  or  of  interviews  on  a 
copy  of  the  interview  protocol.     Again,   summaries  of 
other  types  of  data  collected  may  be  written,  or 
presented  in  tables. 

Reporting  the  results  on  use  or  implementation 
questions  may  be  more  difficult.     Results  of  surveys 
and  interviews  can  be  done  in  a  traditional  manner, 
however,   reporting  results  of  observations  and 
microanalyses  of  data  can  be  done  many  different  ways. 
Frequency  tables  can  be  developed  for  categories  of 
coded  behaviors.     Although  not  an  evaluation  study,  pre 
se,  an  example  of  the  reporting  of  teacher  perceptions 
and  planning  behaviors  reported  in  a  case  study  style 
is  presented  by  Reiser  and  Mory  (1991).  Alternately, 
some  evaluators  build  a  type  of  story  description,  or 
scenario,  of  patterns  they  have  observed.     It  may  be 
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useful  for  evaluators  to  turn  to  descriptions  of 
qualitative  research  in  social  sciences  for  types  of 
methods  to  try  (cf„  Bogdan  &  Biklen,  1982;  Straus, 
1987) . 

Conclusions/Recommendations 

In  conclusion,  alternate  methods  of  conducting 
formative  evaluations  may  be  particularly  useful  and 
crucial  when  dealing  with  highly  innovative  interactive 
technology-based  instruction.     One  key  to  success  is  to 
ensure  that  evaluation  questions  drive  the  choice  of 
methods  for  collecting  data  and  reporting  results - 
Another  is  to  keep  the  evaluation  focussed,  thus 
simple  and  efficient.     Another  factor  in  success  is  to 
use  rigorous  techniques  and  methods  while  experimenting 
with  new  ways  of  conducting  evaluations.  Evaluators 
will  learn  more  about  how  their  innovative  technology 
systems  are  being  used  if  they  are  open  to  what  is 
really  occurring,  but  not  overwhelmed  to  the 
point  that  they  gather  too  much  data,   collect  data 
haphazardly,  or  focus  on  data  items  which  are  so 
ideosyncratic  that  the  results  cannot  be  compared  to 
any  other  data  or  results  of  any  other  studies. 

As  always,  the  main  question  is  whether  students 
learned  using  the  interactive  instruction,   no  matter 
how  attractive  the  "bells  and  whistles." 
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Students'  Attitudes  Toward  Small  Group  CBI: 
A  Question  of  Aptitude 

ABSTRACT 

This  investigation  examined  middle-school  students'  attitudes  toward  and  abilities 
to  benefit  from  small  group  computer-based  instruction  (CBI)  as  a  function  of  their 
academic  aptitudes,  gender,  and  learning  situations.  Two  studies  were  conducted  with 
different  tasks.  Seventy-six  seventh  graders  (39  high  aptitude  and  37  low  aptitude)  from  a 
suburban  school  completed  a  task  designed  for  individual  performance;  while  a  group  of 
56  students  from  an  Inner-city  middle  school  completed  a  task  designed  for  small  group 
performance.  After  completing  the  tasks  designed  for  individualized  performance,  higher 
ability  students  preferred  individualized  CBI  and  lower  ability  students  preferred  small  group 
CBI;  while  the  opposite  set  of  preferences  was  found  for  the  small  group  learning  task.  A 
complex  interaction  was  then  proposed  among  students'  academic  aptitudes,  their 
preferences  for  small  group  or  individualized  CBI,  and  the  learning  situation. 

Introduction 

Foremost  among  many  recent  trends  in  education  has  been  a  surge  of  interest  in 
cooperative  learning.  A  computer  search  of  the  ERIC  system  produced  628  professional 
works  during  the  last  seven  years  on  this  topic  as  compared  to  105  such  works  from  1965 
to  1982. 

Underlying  this  trend  is  the  belief  that  cooperative  learning  situations  can  help 
educational  programs  and  systems  throughout  the  world  meet  some  of  their  pressing 
practical  concerns.  For  example,  current  economic  problems  have  made  it  impossible  for 
the  U.S.  school  systems  to  have  enough  computers  to  meet  the  needs  of  every  student 
(Becker,  1990;  Sutton,  1991).  These  economic  concerns  have  also  led  to  a  greater  disparity 
in  the  accessibility  of  computers  to  students  from  affluent  and  poor  schools  with  students 
from  the  latter  schools  having  very  limited  access  to  this  instructional  medium  (Sutton). 
Cooperative  or  small  group  use  of  computers  can  thus  allow  more  students-especially 
those  from  disadvantaged  communities-to  use  this  instructional  medium. 

Small  group  learning  is  also  expected  to  give  students  a  more  positive  attitude 
toward  the  instructional  process  (Smith,  1987).  Such  claims  about  cooperative  learning's 
enhancement  of  student  motivation,  especially  that  of  marginal  students  appears  throughout 
the  educational  literature  (Glaser,  1985;  Hooper  &  Hannifan,  1991;  Johnson,  Johnson,  & 
Stanne,  1983;  Mensch,  Lew,  Johnson,  &  Johnson,  1986;  Sharan,  1990;  Slavin,  1987). 
Hooper  and  Hannafin  (1991)  have  noted  that  students  working  individually  for  extended 
periods  often  become  lonely,  bored,  or  frustrated  with  their  work.  Also,  individualized  work 
tends  to  make  at-risk  students  feel  more  isolated  from  their  -veers  (Mensch  et  al.).  Small 
group  use  of  computer-based  instruction  (CBI)  should  thus  hdp  students  overcome  these 
problems  of  frustration  and  isolation  as  students  provide  each  other  with  moral  support 
(Hooper  &  Hannafin;  Rysavy  &  Sales,  1991).  It  is  generally  believed  that  helping  students 
to  overcome  such  feelings  helps  students  to  reach  higher  levels  of  academic  achievement 
(Hooper  &  Hannifan;  Rysavy  &  Sales,  1991). 

An  aptitude-by-treatment  interaction  (ATI)  may  exist  in  students'  preference  for  small 
group  learning  (Sharan  &  Shaulov,  1990).  Pyryt  (1991)  has  intimated  that  higher  ability 
students  are  more  oriented  toward  individualistic  learning  situations  than  cooperative 
learning  situations  as  these  students  prefer  competitive  activities.  Lower  ability  students 
seem  to  prefer  cooperative  learning  situations  as  these  learning  environments  provide  them 
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with  needed  structure  and  moral  support.  Hence,  lower  ability  students  would  seemingly 
prefer  and  benefit  from  small  group  CBI  while  higher  ability  students  would  find 
individualistic  CBI  situations  more  beneficial. 

Clark  (1982)  found  evidence  that  suggests  a  different  relationship  between  students' 
abilities  and  preferences  toward  individualistic  and  small  group  CBI.  Lower  ability  students 
were  found  to  prefer  activities  that  promised  them  relative  anonymity  while  higher  ability 
students  tended  to  prefer  the  more  structured  tasks.  According  to  Clark's  evidence,  lower 
ability  students  would  tend  to  favor  individualistic  CBI  over  small  group  CBI  while  the 
opposite  preference  would  be  found  for  higher  ability  students. 

Dalton,  Hannafin,  and  Hooper  (1989)  did  find  further  empirical  support  for  lower 
ability  students'  preference  for  anonymity.  This  investigation,  which  involved  students' 
learning  scientific  materials,  found  a  complex  interaction  among  students'  gender,  ability 
levels,  and  preferences  for  dyadic  or  individualized  use  of  CBI  materials.  Lower  ability  males 
tended  to  prefer  individualistic  CBI  while  their  female  counterparts  preferred  cooperative 
CBI.  They  claimed  that  the  lower  ability  male  students'  desire  for  anonymity  was  due  to  the 
learning  situation  associated  with  their  study.  That  Is,  male  middle-school  students  did  not 
want  other  students  to  realize  that  their  knowledge  of  science  was  limited.  The  reasons  for 
lower  ability  females'  preference  for  cc  iperative  CBI  were  not  discussed.  They  also  failed 
to  mention  other  aspects  of  this  learning  situation,  such  as  the  composition  of  the  groups, 
which  might  have  affected  the  lower  ability  male  students'  desires  for  anonymity.  Finally, 
Dalton  et  al.  did  not  find  any  ATI  effects  in  the  students'  achievement  scores. 

Except  for  the  evidence  provided  by  Dalton  et  al.,  very  little  empirical  data  exist 
regarding  the  possible  ATI  effects  in  students'  preferences  for  and  abilities  to  benefit  from 
small  group  use  of  computer  technology.  Hence,  many  questions  remain  regarding  such 
possible  ATI  effects,  including  their  very  existence. 

Purpose  of  Investigation 

The  primary  purpose  of  this  investigation  was  to  examine  middle-school  students' 
preferences  for  and  abilities  to  benefit  from  either  small  group  or  individualized  CBI  as  a 
function  of  their  academic  aptitudes,  learning  situation  and  gender.  Also  examined  were  the 
students  preferences  for:  (a)  small  group  CBI  versus  not  working  at  the  computer  and  (b) 
working  with  a  partner  versus  working  with  partners  at  terminals.  A  final  issue  Involved 
changes  in  the  students'  preferences  as  a  function  of  their  learning  experiences.  These  later 
issues  have  also  not  been  fully  explored  by  the  research  literature  (see  Shlechter,  1991). 

Research  Approach 

Two  studies  were  conducted  employing  completely  different  learning  situations. 
One  group  of  students  completed  a  science  task  designed  for  individualized  learning  with 
individualized  measures  of  performance  being  utilized.  A  second  group  used  the  computer 
to  complete  a  social  studies  task  desigmd  for  small  group  learning.  The  performance 
measure  for  this  study  was  the  group's  cor  ipletion  of  the  task.  Also,  homogeneous  ability 
groupings  were  employed  for  Study  1  while  both  homogeneous  and  heterogeneous  ability 
groupings  were  employed  for  Study  2. 

Each  study  was  conducted  as  part  of  the  schools'  academic  program.  All  studies 
were  also  conducted  during  regularly  scheduled  class  sessions  in  the  schools'  computer 
laboratories.  The  subjects  were  neither  encouraged  nor  discouraged  from  engaging  in 
cooperative  activities.  Previous  studies  have  shown  that  students  who  are  working  together 
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at  terminals  are  rarely  instructed  to  engage  in  cooperative  behaviors  (see  Littlejohn,  Ross, 
&  Gump  1984;  Shlechter,  1991). 


Study  1 

Method 

Subjects:  This  sample  consisted  of  eighty  seventh  graders  (40  high  and  40  low 
aptitude)  from  a  suburban  middle-school  In  the  greater  Phoenix,  AZ  area.  Half  of  these 
students  had  a  high  aptitude  for  science,  as  determined  by  teacher  nominations,  and  the 
other  half  had  a  low  aptitude.  Four  students  did  not  complete  the  task  leaving  39  high 
aptitude  students  and  37  low  aptitude  students. 

Training  Conditions:  Subjects  were  assigned  to  one  of  three  training  conditions: 
small  group  (four  per  terminal),  dyad  (two  per  terminal),  or  Individual  (by  themselves).  While 
each  condition  had  approximately  the  same  mix  of  high  and  low  ability  students,  students 
were  assigned  to  homogeneous  ability  working  arrangements.  Students  in  the  small  group 
and  dyad  training  conditions  were  also  arranged  into  either  same  and  mixed  sex  groupings 
with  approximately  the  same  number  of  male-male;  female-female,  and  female-male  working 
arrangements. 

Instruments:  A 13  Item  continuing  motivation  questionnaire  was  adopted  from  one 
previously  used  by  Pollock  (1989).  Each  item  involved  the  students  in  responding  to  one 
of  two  preferences,  such  as: 

I  would  prefer  to  work  on  the  computer: 

a.  with  several  partners. 

b.  by  myself. 

Five  of  these  Items  dealt  with  assessing  students'  preferences  for:  (a)  small  group  versus 
individualized  CBI;  (b)  dyadic  versus  quadratic  terminal  arrangements;  and  (c)  small  group 
versus  not  working  at  the  computer.  The  remaining  eight  Items  dealt  with  students'  feelings 
regarding  the  subject  matter  and  regarding  the  use  of  computers. 

The  immediate  and  delayed  post-tests  consisted  of  30  short-answer  completion 
questions,  which  were  previously  used  by  Pollock  (1989).  She  found  that  the  different  tests 
were  psychometrlcally  sound  with  KR21  reliability  coefficients  in  the  70s. 

Procedure:  The  students  completed  a  CBI  instructional  program  on  tarantulas, 
which  was  previously  used  by  Pollock  (1989).  This  lesson  contained  embedded  multiple- 
choice  and  short-answer  completion  questions.  Subjects  who  were  told  to  work  at  their 
own  pace  took  approximately  fifty  minutes  to  complete  this  task.  Subjects,  individually, 
completed  the  attitudinal  questionnaire  and  the  achievement  test.  Six  weeks  later  they  took, 
individually,  the  delayed  post-test. 

Dependent  Variables  and  Data  Analyses 

Preference  Data:  The  dependent  variables  for  the  attitudinal  measures  were  the 
students'  responses  to  Items  dealing  with  small  group  and  individualized  CBI.  Separate  2 
X  2  chl-square  analyses  per  Item  were  conducted  to  determine  the  existence  of  any  aptitude 
difference  In  these  data.  The  cross-break  variables  for  these  different  chl-square  analyses 
were  the  subjects'  aptitudes  (high  or  low)  by  their  responses  to  each  of  the  two  choices  per 
ltem--e.g.,  working  with  a  partner  or  by  themselves. 
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4X2  chl-square  analyses  were  also  computed  to  examine  any  possible  Interaction 
between  aptitude  and  gender.  The  factors  for  these  analyses  were  four  groups  of  subjects- 
(a)  low  aptitude  females;  (b)  high  aptitude  females;  (c)  low  aptitude  males;  and  (d)  high 
aptitude  males-by  response  frequencies  for  each  choice.  These  chl-square  analyses  were 
conducted  upon  the  subjects'  responses  for  working  with  a  partner  and  with  partners. 

All  chl-square  tests  were  computed  for  proportional  differences  In  the  data.  The 
computer  program-SPSSx-was  used  to  compute  these  tests  with  the  alpha  level  being  set 

at  .05. 

Performance  Data:  The  dependent  variables  for  these  data  were  the  subjects' 
scores  on  the  different  tests.  These  data  were  analyzed  by  the  following  ANOVA  model-a 
3X2X2X2  factorial  for  repeated  measures.  As  previously  indicated,  these  factors 
included  three  levels  of  group  size  (small  group,  dyad,  and  Individualized  CBI);  two  levels 
of  gender  (male  and  female),  two  levels  of  scientific  aptitude  (high  and  lower  aptitude 
students);  and  two  levels  of  tests  (immediate  post-test  versus  delayed  post-test),  which 
were  the  repeated  dimension  of  the  ANOVA  model 

Again,  SPSSx  was  used  to  compute  the  analyses.  And  the  alpha  level  was  Let  at 
.05  tor  all  tests. 

Results 

A  significant  relationship  was  found  with  regards  to  aptitude  differences  and 
students'  preferences  for  working  with  a  partner,  Xf  (1 ,  N  =  76)  =  4.53,  p.  <  .03).  Also,  a 
trend  approaching  significance  was  found  with  regards  to  working  with  partners,  Xj  (1 ,  N 
=  76)  =  3.33;  p  >  .07).  Seventy  percent  of  the  lower  ability  students  preferred  working 
on  the  computer  with  an  assigned  partner  rather  than  working  by  themselves.  Only  46%  of 
the  higher  ability  subjects  had  the  same  preference.  Nearly  55%  of  the  lower  aptitude 
students  favored  working  on  the  computer  in  an  assigned  group  of  four  as  compared  to 
only  30%  of  the  higher  aptitude  students. 

There  was  a  marginally  significant  interaction  between  gender  and  aptitude 
regarding  the  subjects'  preferences  for  working  with  a  partner,  X2  (3,  N= 76) =7.35;  p  .> 
.06).  A  substantially  lower  percentage  of  higher  ability  males  favored  working  with  a  partner 
than  did  any  other  group.  However,  this  interaction  was  not  significant  for  the  subjects' 
preferences  for  working  with  partners. 

Several  other  interesting  patterns  emerge  from  the  preference  data.  Subjects 
overwhelmingly  preferred  to  work  with  a  partner  rather  than  with  partners.  They  preferred 
working  In  small  groups  rather  than  not  working  at  a  computer.  An  interesting  aside  is  that 
subjects-regardless  of  aptitude  and  sex-preferred  to  study  something  other  than  insects 
on  the  computer. 

Performance  Data:  The  ANOVA  test  failed  to  find  any  significant  effects  nor 
interactions  associated  with  group  size.  Students  thus  learn  and  retain  comparable 
amounts  of  information  regardless  of  the  CBI  mode  of  presentation. 

STUDY  2 

Method 

Subjects:  The  subjects  were  fifty-six  (38  females  and  18  males)  seventh  graders 
from  Louisville,  KY.  The  principal  indicated  that  this  sample  consisted  of  12  low  aptitude  (4 
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females  and  8  males),  12  average  students  (7  females  and  5  males),  and  32  (27  females 
and  5  males)  high  aptitude  students. 


This  school  is  located  in  a  racially  mixed  working  class  neighborhood  with  a  public 
housing  project  nearby.  According  to  guidelines  in  the  Jefferson  (Kentucky)  County 
Schools,  the  school  has  a  60-40%  black-to-white  ratio. 

Tasks  and  Procedure:  Several  methodological  changes  were  made  from  Study  1. 
As  previously  indicated,  the  students  developed  two  different  spreadsheets  for  use  in  a 
social  science  course.  Each  task  took  two  fifty-minute  class  periods  to  complete.  A  second 
change  was  the  matter  in  which  the  working  arrangements  were  determined.  To  replicate 
daily  computer  usage,  the  students  determined  their  own  working  arrangements  for  the  first 
task.  Nearly  all  of  them  chose  to  work  with  same  sex  partners  rather  than  by  themselves. 
Also,  most  of  these  groups  consist  of  students  with  relatively  similar  academic  abilities.  That 
is,  groups  would  rarely  consisted  of  students  with  high  and  low  aptitudes.  Five  students 
did  choose  to  work  by  themselves.  No  systematic  patterns  were  found  in  the  students' 
desire  to  work  by  themselves  vis-a-vis  aptitude  differences. 

For  the  second  task,  students  were  assigned  to  work  at  a  terminal  with  other 
students.  This  was  done  to  force  all  students  to  work  together  and  with  classmates  who 
were  not  necessarily  their  friends.  Nearly  all  of  the  assigned  groups  for  this  task  consisted 
of  students  with  differing  abilities  and  sexes. 

A  final  set  of  changes  dealt  with  the  preference  questionnaire.  The  subjects 
completed  this  questionnaire  before  and  after  completing  the  computer  lessons.  Also,  the 
questionnaire  was  slightly  changed  to  assess  the  terminal  arrangements  employed  in  this 
study.  Questions  about  working  with  friends  were  added  while  those  about  working  with  a 
partner  were  eliminated. 

Dependent  Variables  and  Data  Analyses:  Thedepc  ndent  variables  for  the  attltudinal 
measures  were  the  students'  responses  to  each  item.  2x2  chi-square  analyses  were 
conducted  for  both  the  initial  and  final  questionnaire  data  with  the  cross-break  variables 
being  low  and  high  aptitudes  by:  (a)  partners  or  self  and  (a)  friends  and  self.  Middle 
aptitude  students  who  were  not  the  focus  of  this  investigation  were  eliminated  for  these 
analyses.  Also,  the  possible  "interaction"  Latween  gender  and  aptitude  was  not  examined 
due  to  the  limited  number  of  females  and  males  in  the  low  and  high  aptitude  groups, 
respectively. 

The  performance  measure  was  the  teacher's  indications  of  the  different  groups' 
abilities  to  complete  the  assignment.  However,  it  was  not  possible  to  analyze  these  data  as 
the  teacher  indicated  that  all  groups  did  complete  the  assignment  with  very  little  difficulty. 

All  chi-square  tests  were  computed  for  proportional  differences  iti  the  data.  SPSSx 
was  used  to  compute  these  tests  with  the  alpha  level  being  set  at  .05. 

,  A  significant  relationship  existed  in  students'  initial  preferences  toward  working  with 
friends--X2  (1,  N  =  44)=  4.53,  p.  <  .03-and  partners  X2  (1,  N  «  44)=  4.93,  p  <  .03.  Three- 
quarters  of  the  low  aptitude  students  had  an  initial  preference  for  working  with  several 
friends  rather  than  working  by  themselves  while  nearly  63%  of  the  higher  ability  students 
had  the  opposite  preference. 
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A  significant  relationship  was  also  found  for  the  subjects'  final  preferences  for 
working  with  partners  or  by  themselves,  Xj;  (1 ,  t£  -  44)  «  6.38,  e  <  .01 .  Nearly  70%  of  the 
higher  ability  students'  preferred  working  with  partners  while  two-thirds  of  the  low  aptitude 
students  had  the  opposite  preference.  A  comparable  but  non-significant  trend  was  found 
for  the  subnets'  preferences  for  working  with  friends  or  by  themselves. 

Additional  chi-square  analyses  Indicated  that  these  reported  relationships  between 
the  initial  and  final  preference  questionnaire  were  significant  for: 

(1)  high  aptitude  student'  preferences  for  working  with  friends, 
X!(1,N  =  64)=  7.63,  c  <  .01. 

(2)  high  aptitude  students'  preferences  for  working  with 
assigned  partners,    (1,  N  =  64)  =5.11,  p  <  .02. 

and  (3)  lower  aptitude  students'  preferences  for  working  with 
assigned  partners,  X2  (1,  N  =  64)  =  6.17,  q  <  .01. 

Hence,  engaging  in  small  group  CBI  did  have  a  significant  effect  on  these  students' 
preferences  for  small  group  or  individualized  CBI.  Several  other  interesting  patterns  emerge 
from  these  data.  One,  subjects-regardless  of  aptitude-overwhelmlngly  preferred  to  working 
in  small  groups  to  not  working  at  a  computer.  Nearly  all  of  them  found  that  these  exercises 
were  either  very  easy  or  easy  to  complete  and  preferred  to  working  with  friends  over 
working  with  partners.  Gender  effects  were  not  found  in  these  data. 

Summary  and  Conclusions 

Several  interesting  trends  have  emerged  from  this  investigation's  data.  One,  as 
previously  described,  differences  were  found  for  most  measures  regarding  students' 
preferences  for  small  group  or  individualized  CBI.  Hence,  a  aptltude-by-treatment  Interaction 
does  seemingly  exist  between  students'  preferences  for  small  group  or  individualized  CBI 
and  their  academic  abilities. 

Two,  this  ATI  effect  does  appear  to  be  a  function  of  "state  factors"  (i.e,  the  learning 
situation)  rather  than  any  inherent  predisposition  for  small  group  or  individualized  CBI.  As 
stated,  the  higher  and  lower  aptitude  students'  preference  data  were  differentially  affected 
by  the  different  learning  situations.  As  found  in  Study  2,  subjects'  preferences  did  change 
as  a  result  of  engaging  in  small  group  CBI.  A  complex  interaction  might  then  exist  among 
students'  academic  aptitudes,  their  preferences  for  small  group  or  individualized  CBI,  and 
state  factors. 

This  complex  interaction  does  not  seem  to  be  contaminated  by  the  previously 
discussed  gender-by-aptltude  interaction  found  in  Study  1.  This  interaction  may  not  have 
been  very  robust  as  it  was  a  trend  approaching  significance.  Also,  comparable  findings 
were  found  for  Study  2's  initial  preference  data,  which  included  many  more  females  than 
males.  And,  this  second  study  dealt  with  examining  students'  preferences  for  working  with 
several  students  rather  than  with  a  partner. 

It  was,  however,  beyond  the  scope  of  this  investigation  to  provkJn  any  clear  insights 
into  the  underlying  reasons  for  this  proposed  interaction.  Perhaps  for  the  individualized 
learning  task,  the  lower  ability  students  felt  that  they  needed  help  and  moral  support  from 
other  students  whereas  the  higher  ability  students  felt  that  other  students  represented 
competition.  Higher  ability  students  might  have  felt  that  "they  could  strut  their  stuff  in  the 
group  task  while  the  lower  ability  students  felt  useless. 
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The  underlying  reason  may  have  been  the  social  composition  of  these  groups. 
Study  1  was  composed  of  homogeneous  ability  groupings  as  compared  to  both 
homogeneous  and  heterogeneous  ability  groupings  for  Study  2.  Perhaps  then,  lower  ability 
subjects  felt  less  threatened  when  working  with  each  other  tiian  when  working  with  higher 
ability  subjects.  Correspondingly  then,  higher  ability  subjects  might  have  felt  less  threatened 
by  competition  when  working  with  lower  ability  subjects. 

There  were  some  indications,  however,  that  such  social  factors  did  not  totally 
explain  the  reported  Influences  of  state  factors.  Study  2's  subjects  manifested  comparable 
preferences  for  working  with  friends  and  assigned  partners.  And  these  subjects  were  more 
likely  to  work  with  friends  in  homogeneous  gender  and  (similar)  ability  groupings  and  with 
assigned  partners  in  heterogeneous  ability  and  gender  groupings. 

Whatever  the  underlying  reasons  for  the  proposed  ATI  effects,  these  preference 
data  do  reflect  learning  situations  that  students  feel  are  best  for  them.  The  veracity  of  such 
feelings  was  not  confirmed  by  this  investigation's  data.  Comparable  amounts  of  learning 
and  retention  were  found  across  the  different  training  conditions  for  the  lower  and  higher 
ability  students  in  Study  1 . 

This  investigation's  preference  data  can  still  be  useful  for  helping  educators  to 
design  their  educational  program.  Study  1's  results  indicated  that  small  group  CBI  should 
consist  of  dyads  rather  than  three  or  four  students  at  a  terminal.  Study  2's  students- 
regardless  of  ability  levels-preferred  small  group  CBI  to  not  working  at  a  computer,  which 
means  that  small  group  CBI  can  be  implemented  in  a  heavily  integrated  urban  school. 
Perhaps  then,  small  group  CBI  can  help  eliminate  some  of  the  inequities  found  with  current 
computer  usage  in  American  schools. 

In  closing,  a  complex  interaction  appears  to  exist  among  students'  academic 
aptitudes,  their  preferences  for  small  group  or  individualized  CBI  and  state  factors. 
However,  important  questions  regarding  the  underlying  reasons  for  and  possible 
instructional  benefits  associated  with  this  interaction  are  left  unresolved.  Conceivably,  further 
insights  into  these  important  questions  will  be  presented  at  upcoming  meetings  of  this 
association. 
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Introduction 

It  is  acknowledged  that  nationwide  schools  are  underfunded  for  technology.  However  it  is  not  enough  to  simply 
throw  more  scarce  education  dollars  into  the  voracious  maw  of  computer  hardware  and  related  technologies  and 
assume  that  all  will  be  well.  There  is  growing  evidence  that  the  instructional  technology  base  currently  installed  in 
many  classrooms  is  being  underutilized*  Simply  providing  teachers  with  additional  computer  technology  is 
insufficient  to  ensure  that  it  will  be  effectively  implemented  as  an  integral  portion  of  the  daily  curriculum* 

Although  much  energy  is  being  directed  towards  the  restructuring  of  schools,  technology  has  not^as  yet,  been  viewed 
as  a  key  element  in  the  success  of  these  efforts*  The  advantages  of  computer  technologies  for  enhancing 
instructional  and  learning  environments  are  demonstratively  obvious*  Why  are  they  not  wholeheartedly  adopted  by 
teachers  and  administrators?  What  are  the  obstacles  which  prevent  teachers  from  incorporating  technology  into  the 
instructional  arena.  There  are  basically  two  categories  of  obstacles:  limited  access  and  ease-of-use. 

Limited  Access 

Increasing  amounts  of  computer  technology  are  being  placed  into  schools*  However,  in  many  instances  this 
technology  is  still  not  directly  accessible  to  the  classroom  teacher*  Frequently  the  computer  technology  has  been 
concentrated  in  limited  numbers  in  computer  labs  and  media  centers*  Teachers/classes  are  scheduled  in  the  lab  only 
two  or  three  times  a  week*  This  pull-out  model  makes  it  difficult  for  teachers  to  integrate  the  classroom  curriculum 
with  the  computer  lab  curriculum 

Even  when  computer  systems  are  distributed  into  the  classroom  teachers  may  find  it  difficult  to  gain  access  to  the 
information  stored  on  the  computer*  When  an  electronic  curriculum  is  supported  by  some  type  of  instructional 
management  system,  the  system  may  often  rigidly  control  how  the  curriculum  may  be  used  or  accessed.  Not  only  is 
instructional  information  often  difficult  to  access  on  the  computer,  but  also  administrative  information.  Computers 
are  capable  of  storing  an  incredible  amount  of  data  on  student  backgrounds  and  needs  which  is  of  legitimate  use  to 
the  teacher.  But  that  information  does  a  classroom  teacher  little  good  when  stored  on  a  computer  accessible  only  to 
the  principal  or  on  the  district  mainframe.  If  there  is  no  management  system  the  teacher  is  left  often  with  even  less 
information  about  how  to  use  and  integrate  a  wide  variety  of  software  packages  from  different  publishers. 

Ease  of  Use 

Teachers  are  heavily  burdened  with  the  constraints  of  managing  the  instructional  environment:  student  evaluation  and 
assessment,  matching  individual  student  needs  with  content,  sequencing  instruction,  recording  keeping,  and  reporting 
student  progress.  All  too  often  the  addition  of  computer-based  technology  is  perceived  as  one  element  too  many  in 
an  already  overtaxed  situation.  Teachers  frequently  feel  they  simply  do  not  have  the  time  or  energy  to  master  and 
integrate  an  instructional  device  which  is  perceived  as  complex  and,  all  too  often,  as  only  marginally  relevant  to  the 
existing  curriculum.  For  those  teachers  who  do  make  the  effort,  the  implementation  of  computer-based  technology 
ranges  from  providing  student  fre  ,-time  use  of  software  which  in  some  way  supplements  the  content  being  taught  to 
the  systematic  use  of  computer-based  tools  which  enhance  the  delivery  and  assimilation  of  instruction. 

Stand  alone  software  may  support  instruction  of  discrete  objectives  but  is  not  intended  to  comprehensively  support 
the  entire  curriculum  of  a  specific  content  area  let  alone  cross-curricular  goals  and  objectives.  Rarely  does  stand 
alone  software  provide  the  flexibility  to  customize  the  instruction  based  on  individual  needs  or  even  monitor  and/or 
report  student  progress.  Integrated  learning  systems  are  designed  to  accomplish  the  above  not  just  for  one  student 
but  for  an  entire  school.  However,  the  down-side  of  this  power  is  that  the  user  interface  for  instructional 
management  is  frequently  perceived  as  intimidatingly  complex. 

Most  computer-based  instructional  management  systems  are  outgrowths  of  older,  larger  computer  systems.  They 
were  designed  to  be  used  by  computer  specialists,  not  classroom  teachers.  Often  these  systems  arc  entirely  text  based 
and  requite  extensive  familiarity  with  computer  operating  systems  and  database  systems.  As  a  result,  they  often 
make  the  job  of  the  classroom  teacher  harder,  not  easier. 

The  Renaissance  Project 

Beginning  in  late  1989,  Jostens  Learning  Corporation  undertook  the  development  of  a  new  generation  of  information 
management  systems  for  public  schools.  Code-named  the  Renaissance  Project,  the  development  effort  has  been 
focussed  on  addressing  a  variety  of  problems  which  include  limited  access  ai.d  ease  of  use.  Efforts  were  at  times 
supported  and  constrained  by  the  need  to  develop  a  final  product  that  was  technicall;'  viable  and  economically  feasible 
for  schools  today.  At  limes,  some  theoretical  avenues  were  not  fully  explored  in  order  to  concentrate  on  the 
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sometimes  more  immediate  and  pragmatic  problem  of  "getting  the  product  out  the  door."  Nonetheless,  there  are 
some  empirically  derived  insights  which  may  be  of  use  to  others  trying  to  address  these  same  issues. 


Renaissance  System  Requirements 

The  development  of  the  Renaissance  Information  Management  System  (RIMS)  began  with  an  attempt  to  clearly 
define  what  barriers  currently  existed  to  using  computer  systems  in  public  schools.  Surveys  were  conducted  in 
dozens  of  school  systems  across  the  United  States.  The  surveys  included  questionnaires  from  classroom  teachers  who 
used  computers  frequently,  teachers  who  only  occasionally  used  computers,  teachers  who  had  never  touched  a 
computer,  school  administrators,  teacher's  aides,  parents,  and  community  volunteers.  Jostens  Learning  education 
service  representatives  who  train  school  staffs  on  using  the  company's  integrated  learning  systems  were  included  in 
the  surveys.  On*  of  the  more  useful  techniques  for  getting  information  on  the  issues  of  implementing  technology 
was  the  analysis  of  the  logs  of  Jostens  Help  Line  staff. 

Yet  another  source  of  information  and  guidance  was  a  National  Advisory  Council  made  up  of  expert  classroom 
teachers,  administrators,  and  computer  specialists  from  across  the  nation.  This  council  initially  met  frequently  to 
help  define  the  requirements  for  the  system,  and  then  regularly  for  the  next  two  years  to  constantly  monitor  and 
review  the  development  of  the  system  itself. 

As  a  result  of  this  extensive  data  collection,  several  hundred  requirements  were  defined  for  the  new  system. 
Classroom  teachers  identified  access  to  information  and  professional  resources  as  amongst  the  most  critical 
attributes.  Teachers  wanted  access  to  other  staff  members,  outside  experts,  curriculum  information,  student  records, 
collaborative  learning  tools,  electronic  references  and  productivity  tools.  They  also  wanted  more  control  over 
curriculum  delivery  and  student  assessment  At  the  same  time  that  educators  were  expressing  the  need  for  more 
control  over  the  system  they  also  requested  a  system  far  more  sensitive  and  adaptive  to  individual  student  needs  and 
conditions-an  "intelligent"  instructional  delivery  system. 

RIMS  is  an  information  management  user-interface  for  an  integrated  learning  system  which  graphically  portrays  the 
school  environment  and  borders  on  that  of  virtual  reality.  The  intent  is  to  provide  users  with  tools  -familiar  in 
appearance-which  easily  allow  for  the  customizing  of  instructional  sequences  and  the  monitoring  of  student 
progress. 

In  trying  to  address  these  requirements  the  classic  Catch  22  of  product  development  immediately  became  apparent- 
how  to  add  more  functionality  while  at  the  same  time  making  the  system  easier  to  use.  Clearly  a  new  type  of 
interface  system  was  needed.  A  variety  of  graphic  interface  systems  (GUI)  including  both  the  Macintosh,  Windows, 
and  Next  operating  systems  were  investigated.  While  all  of  these  constituted  an  significant  improvement  over 
previous  text-based  operating  environments,  the  volume  of  information  in  the  system  soon  became  overwhelming 
using  standard  graphical  interfaces  so  other  more  non-conventional  interface  technologies  were  explored. 

RIMS  Interface 

A  new  technique  in  interface  design  is  the  creation  of  three  dimensional,  all  encompassing  representations  of  physical 
space.  Interfaces  such  as  these,which  grew  out  of  CAD/CAM  technology,  surround  the  user  with  a  metaphorical 
space  that  the  user  can  navigate  through  in  order  to  act  in  a  fashion  that  fits  the  way  they  would  act  if  that  space  were 
real.  This  type  of  graphic  environment  is  often  referred  to  as  a  Virtual  reality"  or  "cyberspace".  The  initial  RIMS 
concept  was  to  create  a  functional  environment  that  would  encompass  an  entire  school  buildingv  in  other  words  a 
"virtual  school". 

However  to  do  so  required  an  incredible  amount  of  raw  computer  processing  power  and  vastly  more  data  spane  than 
would  be  available  on  affordable  school  computers .  A  simpler  frame-based  model  was  selected  as  a  viable 
alternative  (See  Figure  1).  Tins  simplified  h.  jdel  which  employed  extensive  use  of  hypertext  still  allowed  for  the 
communication  and  representation  of  information  in  innovative  svays  but  did  not  require  the  sheer  horsepower  of  a 
complete  virtual  reality  system. 
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Figure  1  The  Renaissance  School  Map 


A  description  of  each  of  RIMS  features  would  be  prohibitively  lengthy  and  so  only  two  modules  which  are 
especially  relevant  to  enhancing  teacher  utilization-Student  Records  and  the  Learning  Path  Editor-are  described 
herein. 

Entrance  to  the  world  of  RIMS  begins  by  selecting  a  room  from  the  school  map  (See  Figure  2). 
Figure  2  The  Renaissance  Classroom 
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The  classroom  display  provides  access  to  a  number  of  information  management  and  productivity  tools:  The  file 
cabinet  is  the  doorway  to  a  variety  of  classroom  managers  nt  features  among  them  student  records  and  the  sequencing 
of  instruction.  There  are  three  basic  tasks  to  be  accomplished*  Students  must  be  entered  into  the  system, 
instructional  sequences  must  be  defined,  and  students  must  be  assigned  to  specific  learning  paths.  It  is  not  necessary 
to  accomplish  these  tasks  in  the  order  given. 

Student  Records 

Student  records  utilizes  a  graphic  metaphor  of  tabbed  file  folders  to  make  working  with  student  information  more 
intuitive.  The  folders  are  accessible  by  clicking  on  the  file  cabinet  which  results  in  a  pop-up  menu  of  record 
selections  (see  Figure  3). 


Figure  3  Access  to  student  records  from  the  file  cabinet 


RIMS  Function  Selection 


Classroom  File  Cabinet 


Learning  Paths 
Report  Generator 
Schedule  Generator 
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Open  i 


[  Cancel 


Vcu  *r*  'jswq  tho  Classroom  Filo  Cabinot 


Access  to  student  records  is  controlled  through  special  security  options  defined  under  the  School  Setup  Module.  Data 
on  student  achievement  is  also  stored  in  Student  Records.  Teachers  or  administrators  with  the  proper  file  privileges 
will  be  able  to  add,  delete,  or  modify  student  records  using  simple  pull-down  menus  or  graphic  buttons  (See  Figure 
4). 


Figure  4  Student  Records 
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After  students  have  been  entered  into  the  system,  they  must  be  assigned  to  specific  instructional  sequences  or 
Learning  Paths  (See  Figure  5),  Students  can  be  assigned  to  Learning  Paths  in  entire  sections,  as  individuals,  or  even 
on  a  temporary  basis. 

EtgllieJi  Learning  Path  assignment 
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The  Learning  Path  Editor 

In  an  effort  to  provide  educators  with  the  power  to  customize  instruction  and  student  assessments  a  tool  called  the 
Learning  Path  Editor  (LPE)  was  created.  RIMS  is  designed  to  ensure  that  students  ha:  4  the  prerequisite  knowledge 
needed  before  being  allowed  to  enter  a  unit  of  instruction.  Teachers  also  wanted  the  system  to  "automatically" 
branch  to  remediation  or  to  enrichment  activities  whenever  appropriate  (See  Figure  6),  Hie  RIMS  system  provides  a 
high  degree  of  sophistication  in  directing  the  activities  of  students  while  also  providing  educators  the  power  to 
ovf  /ride  the  system  at  any  time. 

Figure  6  Renaissance  Learning  Path 
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The  LPE  allows  the  customer  to  easily  construct  branched  instructional  sequences  and  assessments  using  an  object 
oriented  graphic  system.  Required  elements  (assessment,  core  instruction,  remediation,  enrichment,  reference  tools) 
need  only  be  clicked  and  dragged  into  place  from  the  menu  bar  to  define  an  instructional  sequence.  As  the  LPE  was 
developed  it  became  apparent  that  the  sheer  volume  of  materials  available  to  the  user  was  overwhelming.  There  were 
literally  thousands  of  lessons,  tools,  and  tool  templates  available  to  any  teacher  who  wished  to  create  a  instructional 
sequence.  It  would  be  impossible  for  any  teacher  to  become  readily  familiar  with  such  a  large  body  of  material.  An 
underlying  hypertext  system  was  created  which  allowed  teachers  to  browse  through  descriptions  of  any  material  for 
potential  inclusion  into  the  electronic  curriculum. 

Figure  7  Learning  Path  unit  level 
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Learning  Path  Level 


Figure  7  is  a  sample  screen  display  of  an  instructional  sequence  at  the  unit  level .  Descriptions  of  each  curriculum 
component  are  provided  at  the  click  of  the  mouse.  Clicking  on  the  unit  icon  and  selecting  Get  Info  provides  the 
teacher  with  detailed  information  about  the  instructional  content  of  that  specific  unit  (See  Figure  8).  The  general 
objective  is  listed  as  are  each  of  the  lessons  provided  in  the  unit.  Clicking  on  the  Learning  Event  produces  a  more 
detailed  listing  of  the  specific  objectives  for  the  lesson.  As  a  navigational  feature  the  source  icon  is  depicted  in  the 
upper  right  comer. 

Figure  g  Unit  level  information 
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Conclusion 

Jostens  Learning  has  assessed  the  needs  of  educators  and  has  developed  a  user  interface  designed  to  empower  the 
management  of  information  and  technology  access  in  the  classroom  The  pragmatic  limits  and  impl  ications  of 
providing  such  interfaces  for  educators  are  being  explored  in  hopes  that  computer-based  instructional  technologies 
will  be  more  readily  employed  in  schools  to  provide  optimal  instructional  environments  for  students.  The  next  step 
is  to  evaluate  the  effectiveness  of  these  new  tools  in  meeting  the  needs  of  educational  information  management  for 
the  classroom  teacher,  students,  administrators,  and  parents. 
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Objectives 

The  objectives  of  this  stu  y  were  to  develop,  implement  and 
evaluate  the  year  long  project,  Microcomputer  Adaptive  Testing  High- 
Risk  Urban  Students  (MATH-R-US).    The  project  produced  diagnostic 
software  to  meet  the  following  criteria:     (a)  help  students  obtain  high 
school  mathematics  credit  needed  for  graduation,  (b)  motivate  students 
to  learn  mathematics,  (c)  account  for  erratic  student  attendance,  and  (d) 
use  computer  adaptive  testing  as  an  integral  part  of  the  program* 

Theoretical  Framework 
The  theoretical  framework  of  this  study  was  based  on  the 
literature  concerning:     (a)  computer  adaptive  testing  of  computational 
skills,  (b)  assessment  as  part  of  instruction,  and  (c)  computer  assisted 
instruction/testing  with  high  risk  students.     Computer  adaptive  testing 
has  been  recognized  as  an  effective  means  of  assessing  student 
performance  of  computational  skills  (Attisha  &  Yazdani,  1983;  Jankc  & 
Pilkey,  1985;  McDonald,  Bcal  &  Ayers,  1987/1988;  Signer,  1982;  &  Travis, 
1984).    The  advantages  of  computer  assisted  testing  include:  (a) 
flexibility  of  administration  time,  (b)  efficiency,  (c)  test  security,  (d) 
clerical  processing  power,  and  (e)  reduction  in  test  construction  time 
(Lippey,  1973;  Reckase,  1986;  Signer,  1982). 

Recommendations  support  the  need  for  the  continual  integration 
of  assessment  into  classroom  instruction  (Bright,  1988;  Bundcrson, 
Inouye  &  Olsen,  1988;  Fisher,  Berliner,  Filbey,  Marliave,  Cahen  and 
Dishaw,  1980;  &  The  National  Council  of  Teachers  of  Mathematics,  1989). 
Moreover,  the  use  of  diagnostic  data  by  teachers  has  been  positively 
correlated  with  student  achievement  (Bright,  1987;  Lamon  &  Bright, 
1987). 

Evaluation  studies  of  computer  use  in  programs  for  students  in 
danger  of  dropping  out  of  high  school  are  few  (Polin,  1989).  Guerrero 
and  Swan  (1988)  investigated  the  efficacy  of  computer  assisted 
instruction  (CAI)  with  older,  at-risk  students  in  New  York  City.  The 
researchers  reported  improvement  in  student  attitudes  and  pride  in 
their  work.     Johnson  &  Mihal  (1973)  found  that  black  elementary 
students  improved  their  performance  with  computerized  tests.  They 
hypothesized  that  anxiety  increases  when  tests  are  administered  by 
individuals   representing   more   advantaged  populations. 

Project  Description 
The  v,opics  of  the  MATH-R-US  diagnostic  programs  and 
accompanying  practice  sheets  are:     (a)  numeration,  (b)  whole  numbers, 
(c)  fractions,  (d)  decimals,  (e)  percent  ,  and  (f)  measurement.  They 
were  written  to  test  every  required  high  school  general  mathematics 
objective  as  described  in  the  city's  curriculum  guide.     The  project  was 
used  for  an  entire  school  year  by  a  class  in  an  urban  high  school.  The 
school  serves  an  at-risk  predominantly  black  population  with  a  high 
rate  of  absenteeism. 

The  tests  accept  gc     dtive  responses,  rather  than  multiple  choice 
answers.    If  a  student's  answer  is  correct,  the  next  objective  is  tested.  If 
an  answer  is  incorrect,  a  second  question  on  that  objective  is  presented. 
If  that  too  is  incorrect,  than  that  objective  is  considered  missed.  Upon 
completion  of  the  test,  results  are  shown  on  a  computer  monitor,  saved 
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on  a  disk  for  later  retrieval,  and  reproduced  by  a  printer.  Practice 
sheets,  with  answer  keys,  can  then  generated  for  the  missed  objectives, 

The  diagnostic  tests  were  administered  once  a  week  in  the  school's 
computer  lab.    During  the  other  daily  class  meetings,  students  worked  on 
the  computer  generated  practice  sheets.     Students  typically  would  be 
working  on  different  objectives. 

Methods 

Evaluation  was  not  conducted  to  fulfill  external  funding 
requirements.     Instead,  its  purpose  was  to  improve  implementation  and 
furnish  descriptive  data  to  the  classroom  teacher  and  school 
administrators.     In  this  respect,  evaluation  was  applied  according  to  the 
definition  of  Stake  (1967)  which  is  to  describe  inputs,  processes,  and 
outcomes. 

The  program's  author  observed  the  computer  sessions  to  record 
both  teacher  and  student  suggestions.     Recommendations  were 
incorporated  into  the  programs  and  tested  at  the  next  weekly  computer 
class.    Weekly  computer  test  results  were  kept  by  the  teacher,    For  each 
student  he  noted  when  a  test  was  taken,  which  objectives  were  missed, 
and  the  objective  that  was  last  attempted  on  the  test.    The  teacher  decided 
which  tests  to  give  to  each  student  and  whether  or  not  they  should  be 
repeated.    These  records  were  used  to  assess  the  students'  achievement  in 
mathematics. 

A  graduate  student  attended  all  the  computer  sessions  to  record  on 
and  off  task  behaviors.    To  ascertain  student  attitude  toward  their 
instruction,  an  attitude  survey  was  administered  at  the  end  of  the  year. 
The  attitude  instrument  was  based  on  the  Attitude  Toward  Instructional 
Setting  Survey  developed  by  Suydam  (1974).    All  items  are  scored  on  a 
five  point  Likcrt  scale  (strongly  agree,  agree,  neutral,  disagree  and 
strongly  disagree). 

Data  Source 

The  students  were  part  of  a  special  program  in  an  urban  high 
school  serving  predominantly  black  youth.     They  were  targeted  as  the 
most  at-risk  of  dropping  out  before  high  school  graduation.    They  had  a 
history  of  high  absenteeism  and  academic  setbacks. 

Close  to  thirty  students  comprised  the  targeted  group.  However, 
the  actual  group  member  changed  sharply  throughout  the  year  because 
of:    (a)  erratic  attendance,  (b)  students  dropping  out  of  school,  (c) 
students  transferring  to  other  schools,  and  (d)  new  students  being 
admitted  to  the  class.    For  this  reason,  it  was  not  feasible  to  apply 
comparative    testing  procedures. 

Results 

One  finding  of  the  study  war,  intense  student  generated 
competition.    When  students  completed  a  test  with  100%  accuracy,  a 
graphic  of  a  hamburger  appeared  on  the  screen.     The  students  turned 
this  feature  into  a  contest.     Upon  entering  the  computer  lab,  they  raced 
to  sec  who  could  get  the  most  hamburgers  in  that  hour.  Another 
observation  was  the  absence  of  off  task  behaviors.    As  a  result  of  the 
student  competition   for  hamburger  pictures,   students  infrequently  (less 
than  5%  of  observed  computer  class  time)  engaged  in  off  task  behavior. 

An  examination  of  the  attitude  data  was  conducted  to  look  for 
aggregate  responses  and  to  determine  if  there  were  differences 
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according  to  gender.    Only  students  who  attended  at  least  two  consecutive 
months  of  classes  completed  the  survey,    Fifteen  students  met  this 
criteria.     Although  both  males  and  females  responded  positively  to  the 
components  surveyed  overall,  an  inspection  of  the  responses  disclosed 
differences  by  gender.  Results  showed  that  the  males  tended  to 
distribute  their  responses  across  the  five  responses,  while  the  females 
clustered  their  responses  at  the  extremes. 

Means  for  each  item  were  computed  by  gender.    In  order  to 
compare  the  means  for  each  statement,  many  of  the  survey  items  were 
reworded  for  positive  phrasing.     Mean  scores  ranging  from  1  to  1.49 
were  interpreted  to  denote  strongly  agree;  1.50  to  2.49  agree;  2,50  to  3.49 
undecided;  3.50  to  4.49  disagree;  and  4.50  to  5.00  strongly  disagree, 
Attitude  survey  responses  that  reflected  personal  concerns  (fear  of 
operating  the  computer,  receiving  attention,  and  opinion  of  the  teacher) 
were  more  positive  and  stronger  among  the  females,    This  was  also  true 
for  items  that  described  the  classroom  environment  as  conducive  to 
learning.    Table  1  contains  a  list  of  the  attitude  statements  that  were 
rated  differently  by  the  nine  males  and  six  females,    While  both  males 
and  females  expressed  positive  attitudes  about  the  course  components,  it 
is  interesting  that  the  female  responses  reflected  more  confidence  in 
one's  own  abilities. 

An  observation  that  occurred  with  increasing  frequency  was 
student  insistence  that  answers  were  correct,  when  the  computer 
indicated  otherwise.    As  a  result,  the  students  were  very  eager  to  prove 
themselves  to  their  teacher.  This  observation  was  surprising  since 
students,  especially  low  achieving  students,  are  not  inclined  to  review 
examples  they  are  told  are  incorrectly  solved. 

Achievement  in  mathematics  was  studied  by  examining  student 
performance  on  the  diagnostic  tests  during  the  year.     The  unstable 
student  population  demanded  that  this  assessment  look  at  individual 
student  performance  on  repeated  tests.    The  data  in  Table  5  reports  test 
results  for  students  who  took  the  same  topic  test  more  than  once.    Only  17 
of  48  students  met  this  criterion.    While  many  other  students  took  several 
computer  tests,  they  did  not  attend  often  enough  to  have  engaged  in  the 
same  test  more  than  once. 

An  inspection  of  the  computer  math  test  scores  indicated  a 
consistent  improvement  on  retcsting  of  a  topic,  with  a  25%  average 
increase  over  an  earlier  test.    Of  the  43  retests,  only  3  did  not  show  an 
improvement.     The  23  perfect  scores  are  noteworthy,  in  light  of  the 
difficulty  of  many  of  the  test  items.    Table  2  lists  the  more  difficult 
objectives  for  the  topic  tests. 

Educational  Implications 
This  study  supports  the  findings  of  Becker  (1988)  and  Guerrero 
and  Swan  (1988)-     At-risk  high  school  minority  students  can  profit  from 
computer  assisted  instruction  with  increased  motivation,  self- 
confidence,  and  self-discipline.     Even  though  the  students  were  being 
asked  to  solve  100%  of  the  topic  objectives  with  100%  accuracy  (without 
guessing),  they  did  not  feel  that  too  much  work  was  required  for  the 
course. 

It  is  noteworthy  that  the  girls  were  more  definitive  and  self- 
assured  than  the  boys.     Girls  have  historically  been  described  as  more 
computer  anxious  than  boys.     Nevertheless,  unlike  the  boys,  the  girls 
were  undivided  in  their  agreement  that  they  were  confident  they  could 
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operate  the  computer.    They  viewed  the  computer  as  having  a  positive 
effect  on  their  learning  and  expressed  a  sense  of  active  enjoyment. 
These  results  are  considerably  different  from  those  of  Campbell  ;  nd 
Perry  (1988).     Their   esuHs  found  more  minority  stereotyping  of 
computers  as  a  rrnie  uomatiL    The  majority  of  students  in  the  Campbell 
and  Perry  stirfy  had  completed  a  computer  course.    In  contrast,  those  in 
this  study  nere  at-risk  surfcftis  who  used  the  computer  as  a  tool  rather 
than  as  inc  object  of  iau ruction.    An  explanation  of  the  results  of  this 
project  raises  the  possibility  that  older  at-risk,  black  girls  are  not  as 
coiiipuU/c  anxious  us  girls  have  been  reported. 

The  high  risk  black  high  school  girls  in  this  study  seemed  to 
exhib?;   hJ^cr  self  confidence  toward  computers  than  that  found  in 
other  sMidies  that  examined  gender.     Recognizing  that  generalizations 
from  small  samites  are  c-am^rous,  the  results  of  this  investigation  solely 
surest  that  re?w*:^.;  is  fte^ieti  lo  investigate  computer  and  mathematics 
anxiety  rx  hiuv.'s  tc::<;*gc  giifis.     Specificly,  studies  should  examine  the 
matfeeriafic--  and  c r.'iapu' *t  ubiety  of  black,  at-risk,  high  school  girls, 
when  th^  computer  is  used      a  tool,  rather  than  as  the  object  of 
instruction. 

Previous  research  has  leand  a  positive  correlation  of  computer 
access  and  use  witJ*  fnterai  school  achievement  (Becker  (1987); 
Blaschke  (1986);  Gannon  (1986);  Krulik  &  Krulik  (1987).    More  recently, 
the  availability  of  computers  in  middle-  and  upper-income  homes  has 
exacerbated  the  school  equity/access  problem  for  the  black  at-risk 
student.    In  light  of  the  results  of  this  project  and  the  equity  issue, 
technology  affirmative  action  seems  necessary  so  that  low  achievers  will 
have  more  opportunities,  rather  than  fewer,  to  raise  their  levels  of 
achievement. 
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Table  1 

Varying  Mean  Responses  to  the  Attitude  Survey  by  Gender 


ATTITUDE 
CATEGORY 


ATTITUDE  STATEMENT 


MEAN  RATING 
M  F 


Computer 
Confidence 

Computer 
Value 


I  was  afraid  I  couldn't  operate 
the  computer. 

I   eel  the  computer 
waste  of  tinv?-, 

Using  the  computer  was 
boring. 


D 

D 
U 


SD 
SD 


Confidence 
could  do  the 
work. 


Feel 

Important 


Too  much  work  was  required 
for  ^his  class. 

The  slow  worker  didn't  have 
a  chance  in  this  class. 

If  a  student  did  not  learn  in 
this  class,  it  was  his/her  own 
fault. 

Using  the  computer  made  me 
feel  that  I  am  important. 


U 
U 

u 


SD 

D 

D 


Computer  de- 
personalizes 
instruction. 

Enjoyed 
Cocii  outer 


No  one    paid  any  attention  to 
me  when  we  used  the 
computer. 

Time  seemed  to  fly  when  we 
used  the  computers. 


D 


U 


SD 


SA=  Strongly  Agree; 
D  =  Disagree; 


A  =  Agree; 
SD  =  Strongly 


U  =  Undecided; 
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Table  2 

Descriptors  af  Difficult  Test  Items 
TOPIC  OBJECTIVE 


Numeration 
Fractions 

Percent 


Measurement 


Estimating  the  square  root  to  nearest  tenth 

Finding  a  fractional  part  in  a  word  problem 

Adding  unlike  denominators 

Subtracting    unlike  denominators 

Dividing  mixed  numbers 

Solving  ratio  and  proportion  problems 

Finding  percent  when  the  percent  is  a  decimal 
Finding  percent  in  a  word  problem 
Finding  a  number  when  the  percent  is  known 
Finding  percent  of  discount,  given  original  and 
sale  prices 

Finding  the  cost  given  the  percent  of  discount 

Doing  two-step  conversions,  within  a  system 
Applying  knowledge  of  measurement  to  word 
problems 
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Abstract 

This  study  examined  the  relative  effects  of  homogeneous  versus  heterogeneous  ability  grouping  on 
performance  and  attitudes  of  students  working  cooperatively  during  interactive  videodisc  instruction.  After 
two  cooperative  training  sessions,  80  fourth  through  sixth  grade  students,  classified  as  high  and  low  ability, 
were  randomly  assigned  to  treatments.  Students  completed  a  level  II  interactive  videodisc  science  lesson,  an 
achievement  test  and  an  attitude  questionnaire.  The  amount  of  instructional  time  for  each  group  was  also 
recorded.  Results  revealed  that  homogeneous  low  ability  groups  scored  significantly  less  than  the  other  three 
groups,  while  the  difference  between  achievement  of  high  ability  students  in  homogeneous  and  heterogeneous 
groups  was  not  statistically  significant.  Homogeneous  low  ability  groups  consistently  used  more 
instructional  time  than  the  other  groups,  whereas  homogeneous  high  ability  groups  used  the  least  amount  of 
time.  Low  ability  students  in  heterogeneous  groups  had  significantly  better  attitude  scores  than  their  high 
ability  groupmates.  Implications  for  the  collaborative  use  of  level  II  videodiscs  are  discussed. 
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The  Impact  of  Cooperative  Group  Composition  on  Student  Performance 
and  Attitudes  During  Interactive  Videodisc  Instruction 

Educators  have  systematically  attempted  to  develop  new  techniques  and  technologies  for  providing 
effective  instruction  for  all  learners.  Much  of  these  efforts  have  result  in  individualized  instruction  that 
adjusts  the  instructional  sequence  to  the  cognitive  and  affective  needs  of  each  learner.  However,  the  potential 
for  individualized  instruction  may  be  limited  due  to  the  difficulties  associated  with  identifying  individual 
differences  and  translating  them  into  instructional  prescriptions, 

Furthermore,  individualized  instruction  has  its  own  shortcomings,  An  important  shortcoming  is  that 
individualization  often  implies  isolation,  Working  alone  for  long  periods  may  cause  boredom,  frustration,  and 
anxiety.  As  a  consequence  of  this  sterile  approach,  students  may  think  that  learning  is  impersonal,  Secondly, 
individualized  instruction  docs  not  allow  students  to  interact  with  and  learn  from  each  other  because  it  limits 
students  to  the  resources  provided  by  the  learning  environment.  Finally,  individualistic  use  of  emerging 
interactive  technologies  such  as  computers  and  videodiscs  greatly  increases  design  and  utility  costs,  Financial 
implications  are  particularly  obvious  when  instruction  requires  a  work  station  for  each  learner  (Hooper  & 
Hannafin,  1991;  Johnson  &  Johnson,  1986). 

It  seems  that  cooperative  learning  has  the  potential  to  overcome  these  limitations.  Carlson  and  Falk 
(1989),  for  example,  concluded  that  cooperative  groups  can  successfully  use  interactive  videodiscs  and  perform 
better  than  those  working  alone.  Nocll  and  Camine  (1989)  indicated  that  cooperative  videodisc  learning  may 
be  more  efficient  than  individualistic  use  of  this  technology.  Atkins  and  Blissctt  (1989)  reported  that  students 
in  small  groups  spent  much  of  their  time  for  interacting  with  partners.  Similar  results  have  been  reported  for 
computer-based  cooperative  learning  (Dalton,  Hannafin,  &  Hooper,  1989;  Johnson,  Skon,  &  Johnson,  1980; 
King,  1989;  Mevarech,  Silber,  &  Fine,  1991).  Moreover,  comprehensive  research  reviews  show  that  the 
benefits  of  cooperative  learning  are  not  limited  to  achievement  effects.  There  is  strong  research  evidence 
demonstrating  the  affective  benefits  of  working  in  cooperative  groups  (Johnson  &  Johnson,  1989;  Newmann 
&  Thompson,  1987;  Rysavy  &  Sales,  1991;  Sharan,  1980;  Slavin,  1991;  Webb,  1988), 

In  addition  to  establishing  the  efficacy  of  cooperative  learning  for  technology-based  instruction,  die 
researchers  must  identify  factors  which  influence  the  effectiveness  of  learning  in  small  groups.  One  of  tficse 
factors  is  group  composition  with  regard  to  student  ability.  Cooperative  learning  usually  involves 
heterogeneous  grouping.  That  is,  groups  are  formed  by  combi  ,ing  students  of  disparate  ability,  gender,  and 
ethnic  background.  However,  there  is  considerable  disagreement  regarding  the  effects  of  heterogeneous 
grouping  on  performance  and  attitudes  of  students  representing  different  abilities. 

Advocates  of  heterogeneous  grouping  claim  that  there  might  be  some  important  advantages  to  having 
students  from  different  abilities  work  together  on  cooperative  tasks.  They  arg;>  that  while  high  ability 
students  benefit  from  providing  explan;  '  ns  to  partners,  low  ability  students  benefit  from  the  increased 
opportunities  for  support  and  encouragement.  The  results  of  some  experimental  studies  showed  that  students 
of  all  abilities  benefitted  from  participating  in  a  heterogeneous  cooperative  group  compared  to  students  of 
similar  ability  who  worked  alone  (Dalton,  Hannafin,  &  Hooper,  1989;  Gabbert,  Johnson,  &  Johnson,  1986; 
Hooper  &  Hannafin,  1988;  Johnson,  Johnson,  Roy,  &  Zaidman,  1985;  Johnson,  Skon,  &  Johnson,  1980; 
Yager,  Johnson,  Johnson,  &  Snider,  1986). 

Critics  claim  that  heterogeneous  grouping  promotes  personal  gains  at  the  expense  of  others.  Hill 
(1982)  indicated  that  the  performance  of  high  ability  students  on  complex  tasks  may  be  detrimentally  affected 
by  medium  and  low  ability  students.  Beane  and  Lemke  (1971),  however,  found  that  high  ability  students 
benefitted  more  from  heterogeneous  grouping.  Slavin  (1983)  suggested  that  heterogeneous  grouping  may 
offer  few  benefits  to  low  ability  students  because  they  are  simply  given  the  correct  answers  and  do  not  learn 
skills  necessary  to  achieve  when  working  alone.  Goldman  (1965),  on  the  other  hand,  reported  that  students 
working  with  partners  of  higher  abilities  performed  better  than  those  working  with  partners  of  similar  or 
lower  abilities.  Swing  and  Peterson  (1982)  found  that  peer  interaction  in  mixed  groups  enhanced  the 
achievement  of  high  and  low  ability  learners,  but  did  not  htfvc  any  affect  on  the  performance  of  medium 
ability  students.  Webb  (1982)  reached  a  similar  conclusion  that  average  students  in  homogeneous  groups 
showed  higher  achievement  and  received  more  explanations  than  average  students  in  heterogeneous  groups. 

In  short,  the  results  of  cooperative  learning  studies  which  examined  the  effects  of  ability  on  student 
performance  and  attitudes  are  inconclusive.  More  research  is  needed  to  clarify  this  uncertainty.  The  present 
study  attempted  to  extend  these  findings  by  determining  whether  interactive  videodisc  instruction  can  be 
completed  as  effectively  in  a  mixed  small  group  as  in  a  uniform  ability  group. 
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More  specifically,  the  following  questions  were  addressed  in  this  study:  (1)  What  is  the  impact  of 
heterogeneous  and  homogeneous  ability  grouping  on  achievement  of  students  working  cooperatively  on  a 
science  task;  (2)  Does  heterogeneous  or  homogeneous  ability  grouping  influence  differently  the  amount  of 
time  spent  in  a  cooperative  group;  and  (3)  How  do  heterogeneous  and  homogeneous  ability  grouping  affect 
student  attitudes  toward  delivery  system,  subject  matter,  and  group  work. 


A  sample  of  80  fourth  through  si/ft  grade  students  selected  from  a  rural  school  district  in  Minnesota 
participated  in  the  study.  Of  this  total  number,  36  (45%)  were  males  and  44  (55%)  were  females.  The  sample 
included  20  (25%)  fourth  graders,  32  (40%)  fifth  graders,  and  28  (35%)  sixth  graders.  Equa>  number  of 
subjects  were  randomly  assigned  to  treatments;  stratified  for  ability,  gender,  and  ethnic  background. 


Lesson  content.  Subject  have  completed  a  level  II  interactive  videodisc  lesson  about  whales.  The 
basic  lesson  included  four  segments:  (a)  characteristics  of  whales;  (b)  kinds  of  whales;  (c)  behavior  of  whales; 
and  (d)  whales  and  people.  Each  lesson  segment  contained  presentation  sequences  and  relevant  embedded 
questions.  The  lesson  began  by  displaying  a  title  screen  will)  directions.  Leuoicrs  were  then  presented  a  main 
menu  showing  each  of  the  four  segments.  Once  a  segment  was  chosen,  students  could  not  exit  until  that 
particular  segment  had  been  completed.  However,  learners  could  watch  a  segment  as  many  times  as  they 
wished. 

Pretest.  Students'  composite  scores  on  the  Iowa  Test  of  Basic  Skills  were  used  for  assigning  them 
into  high  and  low  ability  groups.  The  median  score  for  the  overall  group  was  taken  as  the  cut-off  point. 
High  ability  students  were  defined  as  those  with  combined  scores  above  the  median,  while  students  with 
combined  scores  below  the  median  were  defined  as  low  ability.  As  a  way  of  reducing  the  classification  error, 
middle  10%  of  the  students  (those  falling  between  45th  and  55th  percentile)  were  not  included. 

Posttest.  The  achievement  posttest  contained  40  items  divided  among  10  true/false,  25  multiple- 
choice,  and  5  matching  questions.  The  KR-20  reliability  coefficient  for  this  test  was  .74,  with  an  average 
item  difficulty  of  .70.  A  typical  item  on  the  posttest  was:  "Mother  whales  feed  or  nurse  their  young  with  (a) 
shrimp,  (b)  milk  in  their  bodies,  (c)  moss". 

Attitude  Questionnaire.  This  instrument  measured  students' reactions  to  cooperative  learning.  It 
included  27  Likcrt-typc  items  divided  equally  among  three  categories:  attitudes  toward  delivery  system, 
attitudes  toward  subject  matter,  and  attitudes  toward  group  work.  Possible  responses  ranged  from  "Strongly 
Agree"  to  "Strongly  Disagree".  The  Coefficient  Alpha  reliability  for  the  questionnaire  was  .82.  A  typical 
item  on  this  instrument  was:  "I  feel  more  comfortable  working  in  a  small  group  than  working  alone". 


Based  upon  their  ability  scores,  students  were  randomly  assigned  to  homogeneous  and  heterogeneous 
cooperative  groups.  Each  heterogeneous  group  had  two  high  and  two  low  ability  members,  while  each 
homogeneous  group  had  four  students  of  the  same  ability.  Prior  to  the  study,  all  subjects  participated  in  two 
training  sessions  on  cooperative  learning.  Students  recorded  the  time  before  starting  the  lesson,  and  again 
upon  completion.  Following  the  instruction,  they  responded  to  an  achievement  posttest  and  an  attitude 
questionnaire  on  individual  basis.  The  entire  experiment  lasted  about  two  hours  for  each  student.  Subjects 
were  excused  from  their  classes  during  the  experiment  and  given  a  token  reward  for  participating  in  the  study. 

Cooperation  Training.  The  main  purpose  of  this  training  activity  was  to  help  students  become  better 
cooperative  learners  on  the  study  tasks.  Two  training  sessions  were  conducted  over  two  consecutive  days. 
Participants  completed  several  exercises  emphasizing  the  basic  principles  of  cooperative  learning.  First,  the 
"Magic  Triangle"  exercise  (Johnson,  Johnson,  &  Holubcc,  1991)  required  students  to  find  the  maximum 
number  of  embedded  triangles  in  a  big  one.  Some  students  completed  this  exercise  in  small  groups,  while  the 
others  worked  individually.  The  result  was  overwhelmingly  better  success  rate  for  those  working  together  in 
small  groups.  This  helped  the  studenLs  draw  the  conclusion  that  working  together  can  be  more  effective  than 
working  alone.  The  students  then  collaborated  in  small  teams  to  work  on  a  rule-generation  exercise  for 
groups  of  similar  figures.  The  rules  to  be  generated  ranged  from  fairly  easy  to  very  complex.  Successful 
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teams  shared  their  answers  and  strategies  with  the  whole  class.  The  class  discussion  then  focused  on  specific 
behaviors  that  were  helpful  (e.g.  explaining)  and  no;  helpful  (e.g.  teasing). 

The  second  day  of  training  activities  began  with  the  "Broken  Circles"  exercise  (Cohen,  1986).  This 
game  emphasized  the  importance  of  positive  interdependence  among  members  of  a  cooperative  group.  Each 
group  was  given  pieces  of  broken  circles.  The  task  was  to  form  at  least  one  complete  circle  for  each  member 
of  the  group,  but  the  students  had  to  exchange  pieces  voluntarily  without  being  asked.  The  students  were  also 
instructed  not  to  talk  and  take  others'  pieces  unless  offered.  This  game  forced  the  group  members  to  pool  and 
share  their  resources  in  accomplishing  the  mutual  goal.  The  fourth  exercise  was  about  correcting  some 
grammar  errors  in  a  short  paragraph.  Such  an  exercise  provided  the  students  with  additional  opportunities  to 
practice  cooperative  skills  on  a  school-related  task.  Finally,  the  students  were  ^sked  to  discuss  the  strengths 
and  weaknesses  of  their  group  work  by  listing  five  advantages  and  disadvantages  of  working  together. 

Treatment.  Students  completed  a  level  II  interactive  videodisc  science  lesson  in  small  groups.  Each 
tutorial-like  presentation  segment  was  followed  by  practice  questions.  Before  responding  to  each  question,  the 
students  discussed  the  options  and  attempted  to  reach  a  consensus  about  the  answer.  Following  their 
response,  they  received  feedback.  When  the  answer  was  correct,  they  were  presented  affirmative  feedback  and 
progressed  to  the  next  item.  If  incoruct,  however,  the  students  were  given  a  second  chance.  After  two 
incorrect  responses,  the  review  of  the  relevant  lesson  segment  was  automatically  provided.  Group  members 
watched  the  review  and  discussed  it  one  more  time.  When  they  have  completed  a  segment,  the  main  menu 
was  displayed  and  the  group  members  selected  the  segment  they  wanted  to  study.  Upon  completion  of  the 
lesson,  the  students  responded  to  the  achievement  posttest  and  the  attitude  questionnaire  individually, 

Design  and  Data  Analysis 

The  study  employed  a  2  by  2  randomized  block  design.  The  first  factor  was  the  type  of  grouping 
(heterogeneous  versus  homogeneous),  and  the  second  ability  (high  versus  low).  The  dependent  variables  were 
achievement,  time  on  task,  and  attitudes  toward  instructional  delivery  system,  subject  matter,  and  team  work. 

Results 

Achievement 

Means  and  standard  deviations  for  achievement  scores  are  presented  in  Table  1,  High  ability  students 
(M=^0,13)  performed  better  on  the  posttest  than  low  ability  students  (M=25,  65).  Also,  the  overall  mean 
score  for  heterogeneous  groups  was  slightly  higher  than  the  overall  mean  score  for  homogeneous  groups 
(M=28.63  and  M=27,15,  respectively). 

Table  1 

Means  and  Standard  Deviations  for  Posttest  Scores 


.CICQllfiinB 

Ability  Homogeneous  Heterogeneous  Total 


High 


Mean: 

3C.45 

29.80 

30.13 

SD: 

2.61 

3.02 

2.80 

N: 

20 

20 

40 

Low 

Mean: 

23.85 

27.45 

25.65 

SD: 

5.15 

3.79 

4.82 

N: 

20 

20 

40 

Combined 

Mean: 

27.15 

28.63 

27.89 

SD: 

5.24 

3.59 

4.52 

N: 

40 

40 

80 

Two-way  ANOVA  results  yielded  a  significant  main  effect  for  student  ability  E(1,76)=28J9, 
B<.001.  The  main  effect  for  ty;.x;  of  grouping,  on  the  other  hand,  was  not  significant  E(l,76)~3.06,  p>.084. 
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The  interaction  between  ability  and  group  composition  was  also  significant  £(1 , 76)-6.36,  ji<.014<  Figure  1 
projects  these  results. 
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Figure  1.  Interaction  Between  Ability  and  Group  Composition  on  Posttest  Scores 

One-way  ANOVA  results  revealed  that  the  difference  between  the  achievement  of  high  ability 
students  in  cipher  groups  was  not  significant  F(l,38)=0.53t  j2>.471.  However,  the  difference  between  the 
achievement  of  low  ability  students  was  significant  in  favor  of  heterogeneous  grouping  F(l, 38)-6.33, 
p<.016. 

XifflS  OP  la&k 

Means  and  standard  deviations  for  time  on-task  are  shown  in  Table  2,  High  ability  students  used  less 
less  time  (M=33.80)  than  low  abiiity  students  (M=40.60).  The  overall  mean  for  heterogeneous  groups  was 
smaller  than  the  overall  mean  for  homogeneous  groups  (M-34.60  and  M=39.80,  respectively). 

Tabic  2 

Means  and  Standard  Deviations  for  Instructional  Time 


Ability 

Grouping 

Homogeneous 

Heterogeneous 

Toial 

High 

Mean: 

33.00 

34.60 

33.80 

SD: 

6.05 

4.59 

5.37 

N: 

20 

20 

40 

Low 

Mean: 

46.60 

34.60 

40.60 

SD: 

1.90 

4.59 

7.00 

N: 

20 

20 

40 

Combined 

Mean: 

39.80 

34.60 

37.20 

SD: 

8.19 

3.53 

7.08 

N: 

40 

40 

80 
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Two-way  ANOVA  results  yielded  a  significant  main  effect  for  both  ability  E(1.76)=  44.87,  jj<0()() 
and  type  of  grouping  f(1.76)=26.24,  p<001.  The  interaction  effect  was  also  significant  E0.76)=44.87, 
p<000.  Figure  2  shows  these  relationships. 


Time 


50  ■ 
45  ■ 
40  - 
35  - 


Low 


High 


_  n  ■  1  

Homogeneous  Heterogeneous 
Grouping 

Figure  2.  Interaction  Between  Ability  and  Group  Composition  on  Instructional  Time 

One-way  ANOVA  results  for  the  follow  -up  comparisons  revealed  that  the  difference  in  the  amount  of 
instructional  time  for  high  ability  students  in  either  groups  was  not  significant  E(l,38)-0.89,  i2>.352.  On 
the  other  hand,  the  difference  between  low  ability  students  was  significant,  suggesting  the  efficiency  of 
heterogeneous  grouping  E(l,38)=l  16.52,  £<.001 . 

Attitudes 

Means  and  standard  deviations  for  attitude  scores  are  reported  in  Table  3.  Overall,  the  mean  attitude 
score  for  low  ability  students  was  higher  (M=88.47)  than  the  mean  attitude  score  for  high  ability  students 
(M=85.03).  Similarly,  heterogeneous  ability  grouping  resulted  in  higher  mean  altitude  score  than 
homogeneous  grouping  (M=87.30  and  M=86.20>  respectively). 

Table  3 


Means  and  Standard  Deviations  for  Altitude  Scores 

Cxouping 

Ability 

Homogeneous 

Heterogeneous 

Total 

High 

Mean: 

87.30 

82.75 

85.03 

SD: 

9.54 

12.78 

11.37 

N: 

20 

20 

40 

Low- 

Mean: 

85.10 

91.85 

88.47 

SD: 

9.94 

10.96 

10.88 

N: 

20 

20 

40 

Combined 

Mean: 

86.20 

87.30 

86.75 

SD: 

9.68 

12.62 

11.19 

N: 

40 

40 

80 
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Two-way  ANOVA  result;  yielded  no  significant  main  effect  for  ability  E(176)=2.01,  j2>.16()  and 
type  of  grouping  E(  1,76)^0,20,  £>.652,  However,  the  interaction  effect  was  significant  EO,76)=5.40, 
I2<.022,  We  also  tested  whether  there  was  a  significant  difference  between  subcategories  of  the  questionnaire. 
Neither  the  main  effect  for  subcategories  £(!  76)=1.37,  jp>.256,  nor  the  interaction  effects f(l,76)=0. 50, 
0>.61O  (ability  by  subcategory);  £(1/76)^0,98,  x>>.376  (grouping  by  subcategory);  andEO/76)=2.34,  p>,099 
(ability  by  grouping  by  subcategory)  were  significant.  This  situation  is  reflected  in  Figure  3. 

Scores 
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EiftUfiLl-  Interaction  Between  Ability  and  Group  Composition  on  Attitude  Scores 

One-way  ANOVA  results  for  follow-up  comparisons  revealed  that  the  difference  between  the  mean 
altitude  score  for  high  ability  students  in  either  groups  was  not  significant  EO,38)=1.89,  £>.177.  On  the 
otter  hand,  the  difference  between  the  mean  attitude  score  for  low  ability  students  was  significant  in  favor  of 
heterogeneous  grouping  F(l  ,38)=  5.66,  j2<.022. 

Discussion 

This  study  examined  the  effects  of  homogeneous  versus  heterogeneous  ability  grouping  on  student 
performance,  time  on  task,  and  attitudes  toward  delivery  system,  subject  matter,  and  group  work  during 
interactive  videodisc  instruction  in  elementary  science.  At  the  end  of  the  lesson,  the  students  completed  a 
posttest  and  an  attitude  questionnaire. 

Results  of  the  study  showed  that  heterogeneous  ability  grouping  was  not  detrimental  for  achievement, 
regardless  of  students'  ability  levels.  There  was  no  significant  difference  in  the  performance  of  high-ability 
students  in  either  groups.  However,  homogeneous  ability  grouping  was  detrimental  for  the  achievement  of 
low-ability  students.  This  is  consistent  with  the  findings  of  some  other  studies  (Goldman,  1965;  Hooper  & 
Hannafin,  1988;  Johnson  &  Johnson,  1989;  Swing  &  Peterson,  1982;  Webb,  1982).  One  possible 
explanation  for  this  result  would  be  that  homogeneously  grouped  low  ability  students  may  not  be  capable  of 
supporting  each  other's  learning  needs.  They  may  also  perceive  their  partners  as  not  being  able  to  find  and 
explain  better  solution  proposals  to  the  difficulties  of  the  group.  On  the  other  hand,  high-ability  students  in 
homogeneous  groups  may  overestimate  each  other's  intellectual  power.  As  Webb  (1988)  suggests,  they  may 
mistakenly  assume  that  everyone  in  the  group  understands  the  task  and  therefore  may  interact  less  efficiently. 
Future  research  should  test  the  validity  of  these  speculations  by  comparing  peer  interaction  within 
homogeneous  groups. 

Although  homogeneous  high  ability  groups  spent  less  instructional  time  than  heterogeneous  groups, 
the  difference  was  not  statistically  significant.  However,  low  ability  homogeneous  groups  consistently  used 
more  time  on  task  than  heterogeneous  groups.  This,  in  general,  supports  the  notion  that  heterogeneous 
groups  tire  more  efficient  for  less-able  students  than  homogeneous  groups  (Hooper  &  Hannafin,  1988;  Webb 
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&  Gillian,  1983).  The  reason  might  be  that  members  of  heterogeneous  ability  groups  show  higher 
understanding  and  employ  better  learning  strategics  in  accomplishing  the  mutual  task.  As  a  result  of 
interacting  with  their  more-able  partners  in  heterogeneous  groups,  low-ability  students  may  acquire  better 
problem  solving  skills.  The  use  of  more  efficient  strategies  can  accelerate  slow  learners  in  heterogeneous 
groups  and  reduce  the  time  on  task.  On  the  other  hand,  homogeneous  low  ability  groups  cannot  take 
advantage  of  diversity  which  helps  them  avoid  typical  mistakes  and  save  time. 

The  results  of  the  present  study  also  demonstrated  that  heterogeneous  ability  grouping  generates  more 
positive  attitudes  toward  instructional  delivery  system,  subject  matter,  and  team  work  than  homogeneous 
grouping.  The  difference  is  particularly  noticeable  for  low  ability  students  in  mixed  groups,  without  an 
accompanying  decrement  for  their  high-ability  groupmates.  Homogeneous  low  ability  groups  reported  the 
lowest  attitude  scores  than  the  other  three  groups,  but  the  difference  between  high  ability  students  in  either 
groups  was  not  significant.  This  agrees  with  the  findings  of  previous  research  (Hooper  &  Hannafin,  1991; 
Johnson,  Johnson,  &  Maruyama,  1983;  Webb,  1983).  Perhaps,  low-ability  students  in  heterogeneous  groups 
feel  more  supported  and  satisfied  than  other  students.  They  may  also  feel  privileged  because  their  high  ability 
groupmates  are  always  available  to  help  them.  This  special  care,  encouragement,  liking,  and  mutual  trust  in 
heterogeneous  groups  may  promote  better  social  relationships  among  group  members. 

Conclusion 

Although  emerging  interactive  technologies  are  gradually  taking  place  in  schools,  educators  arc  still 
facing  a  serious  dilemma.  They  arc  cither  to  find  more  effective  way*  f  using  these  technologies  in  the 
classroom  or  to  limit  the  access  by  large  groups  of  students.  Interactive  videodisc  is  no  exception. 
Fortunately,  however,  new  methods  are  being  successfully  developed  for  the  effective  use  of  interactive 
videodisc  systems.  Cooperative  learning  is  one  of  them. 

In  general,  advocates  of  cooperative  learning  recommend  that  students  be  grouped  heterogcneously. 
However,  the  benefits  of  heterogeneous  grouping  have  not  been  clearly  established.  Some  claim  that 
heterogeneous  ability  grouping  supports  the  needs  of  one  group  at  the  expense  of  another.  There  is  also 
substantial  evidence  suggesting  that  heterogeneous  grouping  is  effective  for  all  students.  The  results  of  this 
study  supports  the  latter  view.  That  is,  heterogeneous  b  juping  is  more  effective  and  efficient  than 
homogeneous  grouping.  Thus,  instructional  designers  and  classroom  teachers  should  consider  using 
heterogeneous  groups  when  designing  cooperative  videodisc  environments. 

It  is  a  fact  that  homogeneous  grouping  is  still  a  common  practice  in  public  schools.  Unfortunately, 
educators  tend  to  continue  this  trend.  Therefore,  future  research  should  focus  on  developing  more  effective 
interventions  within  homogeneous  learning  groups.  One  of  the  potential  variables  that  should  be  investigated 
further  is  the  impact  of  exercising  learner  control  during  cooperative  group  work.  Future  research  should  also 
examine  the  effects  of  manipulating  group  size. 
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Abstract 

The  effects  of  English  captioning  used  as  knowledge  of  results 
feedback  during  English  listening  comprehension  practice  for 
Taiwanese  students  were  investigated.  Seventy-two  college  freshmen 
at  National  Taiwan  Normal  University  participated  in  the  study,  using 
an  interactive  videodisc  program  one  hour  every  two  weeks  over  ten 
weeks.  Two  versions  of  the  program,  one  with  captioning  and  one 
without,  were  used.  The  effects  on  listening  comprehension  were 
measured  by  both  a  treatment  content  specific  listening  comprehension 
test  and  the  listening  comprehension  sub-test  of  the  Test  of  English  as  a 
Foreign  Language  (TOEFL).  The  results  indicate  that  subjects  in  the 
captioning  treatment  had  a  significantly  higher  score  on  the  treatment 
content  specific  listening  comprehension  test.  There  was  no  significant 
difference  between  the  groups  on  the  TOEFL  listening  comprehension 
test  which  measures  general  comprehension  skills.  In  addition, 
English  reading  ability  did  have  a  significant  impact  on  listening 
comprehension  performance  with  those  subjects  in  the  above  average 
group  preforming  better  than  those  in  the  below  average  group.  These 
results  suggest  that  using  captioning  for  specific  content  will  improve 
the  learners  comprehension  within  that  content.  It  is  possible  that 
exposing  students  to  a  wide  variety  of  content  areas  using  captioning 
may  have  a  positive  impact  on  general  listening  comprehension  skills. 
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While  many  Taiwanese  students  have  studied  English  for  at 
least  seven  years,  they  still  cannot  readily  understand  movies,  TV 
programs,  or  class  lectures.  In  addition,  many  students  who  take  the 
TOEFL  (Test  of  English  as  a  Foreign  Language)  receive  a  low  score  on 
the  listening  comprehension  part  of  the  test  This  may  be  because  they 
have  English  reading  and  grammar  classes  but  no  listening  classes  in 
high  school,  college,  or  university.  Furthermore,  many  teachers  still 
use  the  traditional  grammar-translation  method  to  teach  English. 
Thus,  many  students  are  unable  to  effectively  communicate  with 
native  speakers. 

Both  language  teachers  and  students  in  Taiwan  tend  to  neglect 
the  importance  of  listening  comprehension  skills.  They  do  so  because 
they  focus  on  a  final  goal  of  reading.  They  fail  to  realize  the  necessity 
for  developing  listening  comprehension  skills  as  a  prerequisite  to 
developing  speaking  skills.  This  basic  orientation  is  reflected  in  the 
often-heard  question,  "Do  you  speak  English?"  Obviously,  one  cannot 
speak  a  language  unless  one  can  understand  it  (Chastain,  1988). 

Although  listening  comprehension  is  of  primary  importance  in 
second  language  acquisition  (Krasen  &  Terrell,  1983;  Krasen,  Terrell, 
Ehrman,  and  Herzog,  1984),  many  contemporary  foreign  language 
educators  have  neglected  its  importance  (Krasen  et  al,  1984;  Paulston  & 
Brunder,  1976;  Rivers,  1981).  This  neglect  may  stem  from  the  fact  that 
listening  is  considered  a  receptive  or  passive  skill  and  from  the  belief 
that  merely  exposing  the  student  to  the  spoken  language  is  adequate 
instruction  in  listening  comprehension  (Call,  1985).  However,  foreign 
language  learners  must  know  the  language  rules  before  they  can 
produce  language,  "competence  must  precede  performance"  (Chastain, 
1988,  p.  91).  Thus,  Chastain  emphasizes  that  "comprehension 
activities,  such  as  those  students  perform  in  listening  and  reading, 
must  precede  productive  activities,  that  is,  speaking  and  writing"  (p. 
92).  In  other  words,  listening  and  reading,  the  receptive  skills,  are 
methods  for  developing  and  expanding  competence. 

Postovsky  (1974)  found  that  pronunciation  of  Russian  is  better 
when  learners  are  encouraged  to  listen  and  not  speak  during  the  earlv 
stages  of  instruction.  Morley  (1985)  also  proposed  that  students  should 
not  be  asked  to  produce  language  before  they  possess  enough 
knowledge  to  make  production  possible.  In  other  words,  foreign 
language  learners  should  learn  to  listen  and  comprehend  first,  then 
speak.  That  is  because  early  listening  instruction  allows  learners  to 
internalize  representations  of  sound,  providing  a  better  basis  for 
producing  and  evaluating  their  own  speech  production  (Carroll,  1986). 

Listening  comprehension  is  characterized  as  a  highly-complex, 
problem-solving  activity  that  can  be  broken  down  into  a  set  of  subskills 
(Byrnes,  1984).  Two  of  these  skills  are  described  by  Rivers  (1971)  as  the 
recognition  of  component  parts  of  the  language  (words,  phrases,  and 
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clauses)  and  the  memorization  of  these  elements.  Recognizing 
linguistic  elements  is  not  sufficient  for  comprehending  what  is  heard. 
Listeners  must  be  able  to  retain  these  elements  in  short-term  memory 
long  enough  to  interpret  what  the  elements  mean  (Call,  1985).  In  fact, 
for  language  learners,  understanding  the  spoken  word  poses  more 
obstacles  than  speaking,  because  in  generating  speech,  learners  can 
control  the  scope  and  difficulty  of  words  and  expressions.  Listeners, 
however,  encounter  and  decode  unfamiliar  messages  (Wipf,  1984), 

Rivers  and  Temperley  (1978)  suggest  that  short-term  memory  for 
foreign  language  words  is  often  overloaded,  causing  words  to  be  purged 
before  they  can  be  organized  and  interpreted.  Thus,  even  though 
second  language  learners  may  be  able  to  recognize  each  word  the 
speakers  said,  they  may  not  remember  lengthy  utterances  long  enough 
to  interpret  them.  Therefore,  this  study  used  short  clips  in  order  to 
allow  the  learners  to  process  information. 

One  of  the  most  difficult  problems  students  have  in  developing 
listening  comprehension  is  fear  (Chastain,  1988).  They  are  afraid  that 
they  will  be  unable  to  catch  every  word  that  the  speaker  says.  They  are 
also  concerned  that  they  may  not  be  able  to  answer  the  teacher's 
questions  if  they  do  not  comprehend  every  word  and  every  sound. 

Ur  (1984)  argues  that  there  are  some  other  major  problems:  (1) 
Foreign  language  learners  do  not  perceive  certain  English  sounds 
because  these  do  not  exist  at  all  in  their  native  language.  For  example, 
Chinese  students  have  difficulty  pronouncing  and  distinguishing  the 
difference  between  /r/  and  /l/  sounds.  (2)  The  English  systems  of 
intonation,  stress,  and  rhythm  can  interfere  with  the  foreign  learner's 
understanding  of  spoken  English.  For  example,  a  falling  tone  on 
"Pardon  me"  means  "Excuse  me",  while  a  rising  tone  means  "Would 
you  please  repeat  what  you  said?".  (3)  Prediction  is  difficult  for  the 
foreign-language  learner.  If  he/she  can  guess  what  is  going  to  be  said 
next,  he/she  will  be  much  more  likely  to  understand  it  well.  (4)  While 
it  is  obvious  that  a  learner  would  probably  not  understand  a  word 
he/she  has  not  learned  yet,  the  learner  also  needs  to  know  that  certain 
expressions  are  common  in  colloquial  English,  but  taboo  in  formal 
style.  (5)  If  a  native  speaker  knows  what  he/she  is  going  to  say  next, 
his/her  utterances  are  often  delivered  at  a  tremendous  rate.  The 
foreign  language  learner  simply  does  not  have  the  time  to  search  long- 
term  memory  for  their  meaning.  (6)  Many  learners  who  are  used  to 
the  accent  of  their  own  teachers  or  friends  are  frustrated  when  they 
find  they  have  difficulty  understanding  someone  else's  accent.  For 
example,  some  learners  can't  understand  the  British  English  accent,  or 
the  southern  accent  of  the  United  States  of  America. 

Donaldson-Evans  (1981)  maintains  that  there  are  two  other 
common  causes  that  hinder  comprehension.  First,  second  language 
learners  lack  a  sufficient  amount  of  cultural  and  linguistic  knowledge. 
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Speakers  and  listeners  do  not  have  to  share  identical  backjrounds,  but 
the  listener  must  share  enough  knowledge  to  permit  comprehension 
of  the  speaker's  message.  Second,  the  listener  does  not  know  a  large 
quantity  of  vocabulary.  Listeners  do  not  have  to  know  the  meaning  of 
every  word,  since  vocabulary  and  idiomatic  expressions  are  the  key  to 
comprehension.  They  comprehend  better  if  they  can  recognize  enough 
words  to  establish  a  context  for  making  rational  guesses. 

One  way  to  overcome  some  of  these  problems  might  be  the 
presentation  of  verbatim  "captions"  and  visual  cues  along  with  the 
aural  information  to  be  understood.  The  National  Captioning  Institute 
(NCI)  points  out  that  "many  viewers  without  hearing  impairments 
find  ordinary  subtitles  ...  distracting"  (Farrel,  1984).  Yet,  Education 
Week  (1990)  and  Price  (1983)  report  that  students  who  are  learning 
English  as  a  second  language  can  significantly  improve  their  language 
skills  by  watching  closed-captioned  television  programs.  They 
conclude  that  "the  viewers  of  closed-captioned  TV  are  better  able  to 
understand  word  meaning  and  pronunciation,  recognize  spellings  and 
grasp  concepts  much  more  quickly  than  by  reading  alone"  (Educational 
Week,  1990,  p.  10).  There  has  been  little  systematic  research  examining 
the  feedback  effect  of  captions  on  the  learning  of  foreign  languages  in 
interactive  videodisc  systems.  Little  is  known  about  how  normal 
hearing  viewers  loam  from  captioned  material.  It  is  only  as  we 
understand  more  completely  the  learning  process  of  second-language 
learners  that  we  can  more  appropriately  design  and  adapt  instruction  to 
meet  their  needs. 

Along  with  the  cues  captioning  may  provide,  movies  can 
provide  the  body  language,  expression,  and  contextual  cues  difficult  for 
teachers  to  duplicate  within  foreign  language  classrooms.  Chastain 
(1988)  argues  that  movies  and  plays  are  excellent  sources  of  listening 
comprehension  materials  when  the  teacher  can  choose  appropriate 
content  for  use  in  class.  This  is  important  because  in  Taiwan  there  is  a 
lack  of  qualified  teachers  or  experienced  native  speakers  tr  Leach 
listening  comprehension  of  English  as  a  second  language. 

Schneider  and  Bennion  (1983)  maintain  that  videodisc-based 
materials  can  provide  highly  motivating  course  segments  for  second 
language  learning  and  fit  in  well  with  modern  theories  of  language 
acquisition.  Many  studies  have  indicated  that  the  interactive  videodisc- 
is  an  effective  medium  for  improving  listening  skills  (Javetz,  1986; 
Kim,  1987;  Larson,  1984;  Schitai,  1989;  Tighe  &  Zufelt,  1988;  Wyait, 
1984),  because  the  interactive  video  allows  students  to  listen  and 
obs<  rve  the  speaker's  body  language,  facial  expressions,  and  contextual 
environment.  It  allows  the  students  to  control  the  program  in  order  to 
meet  ther  pace  and  to  replay  any  unclear  message  immediately. 
Moreover,  interactive  videodisc  is  a  cost-effective  medium  (Fletcher, 
1989;  West,  1989). 
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Interactive  video  allows  a  teacher  to  create  an  environment 
beneficial  to  learning  in  a  variety  of  areas  (Sherwood,  Kinzer, 
Hasselbring,  Bransford,  Williams,  &  Goin,  1987).  Many  people  have 
repurposed  current  popular  feature  films  on  videodisc  to  develop 
instruction.  Allen  (1986)  maintains  that  a  repurposed  film  or  video 
contains  two  types  of  components  that  are  not  present  in  the  original: 
(1)  algorithms  that  bring  the  video  under  the  control  of  the 
instructional  system  or  the  user;  (2)  addition;* I  messages,  such  as 
computer  text  and  graphic  displays  that  provide  supplementary 
content. 

These  programs  are  based  on  three  theoretical  principles:  (1)  the 
importance  of  a  contextually  rich  learning  environment;  (2)  the  use  of 
instructional  media  to  direct  students'  activities;  and  (3)  the 
development  of  conceptual  tools  that  students  might  use  and  transfer 
to  a  variety  of  settings  (Sherwood  et  al.,  1987).  They  add  that  there  are 
several  advantages  in  adapting  videodisc-based  movies  for  instruction: 
(1)  the  availability  of  material  without  the  productiem  expense  and 
time  delay  of  producing  original  video;  (2)  the  high  production  quality 
of  these  movies;  (3)  the  "naturally"  occurring  problem  situations  fouhd 
in  the  movies,  and  (4)  the  motivational  aspects  of  popular  movies. 

The  characteristic  of  interactive  video  important  for  this  study  is 
the  ability  to  add  computer  generated  messages  to  the  cont<  xtually  ri'ih 
environment  provided  by  the  video.  This  ability  allows  the  system  to 
provide  very  specific  feedback  about  the  correctness  oi  the  learner's 
interpretation  of  the  spoken  English  .rom  the  film.  This  type  oi 
feedback  is  known  as  knowledge  of  results  (KR)  feedback. 

Feedback  is  an  important  design  feature  in  any  instruction.  In 
the  nine  events  of  instruction  model,  Gagne,  Briggs,  and  Wager  (1988) 
emphasize  that  it  is  essential  to  provide  information  to  the  learners 
about  the  correctness  of  their  performance.  However,  feedbaek  is  the 
least  understood  instructional  strategy  (Cohen,  1985).  Results  of 
feedback  studies  are  contradictory.  For  example,  Kulik  and  Kulik  (1988) 
conducted  a  meta-analysis  on  feedback  timing  and  verbal  learning  and 
found  that  immediate  feedback  is  more  effective  thaa  delayed  feedbaek 
in  actual  classroom  quizzes  and  real  learning  materials.  However, 
"experimental  studies  of  acquisition  of  test  content  have  usually 
produced  the  opposite  result"  (Kulik  and  Kulik,  1988,  p.  79). 

Knowledge  of  results  feedback  (KR)  is  an  important  source  oi 
information  for  skill  learning,  although  it  is  not  always  a  necessary 
condition  (Swinnen,  Vandenberghe,  &  Van  Assehe,  1986).  Swinnen  el 
al  (1990)  argue  that  knowledge  of  results  feedback  generates  guidance- 
like  effects  from  feedback  during  practice.  They  also  maintain  that  "KR 
has  a  strong  informational  capability  to  guide  or  direct  p^rformnnce 
when  it  is 'present,  helping  the  learner  to  maintain  performance  at  or 
near  the  target"  (p.  715).  Moreover,  Cohen  (1985)  states  that  KR  Ml-  v. 
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the  kvrnor  to  confirm  tlf 1  accuracy  of  their  responses  in 
comprehending  and  mastering  the  material,  In  addition,  ,fit  may  be 
necessary  to  confirm  every  correct  response  with  KR  so  that  a  student 
feels  comfortable  with  h:s  or  her  mastery  of  the  material"  (p.  35). 

However,  Cohen  (1985)  indicates  that  feedback  messages  that  are 
too  long  ai  d  time-consuming  can  slow  down  the  pace  of  instruction 
and  cause  a  confident  student  to  feel  impatient  and  frustrated, 
Swinnen  et  ai,  (1990)  hold  the  same  point  of  va:vv   Th^y  ar^ue  that 
"KR  may  at  the  same  time  prodice  certain  negative  "s.Je  eKects"  that 
interfere  ia  various  ways  with  longer  term  learning  such  as  would  be 
required  tor  y>erformance  on  a  test  of  retention  or  transfer"  (p.  715), 
They  describe  the*  nogath  e  effects  as;  (1)  In  simpler  tasks,  frequent  KK 
ercourigns  the  learner  to  change  behavior  frequently  in  an  ifemnt  to 
eliminate  errors.  Some  of  the  error  cone*  Hons  are  verv  small  ana  -hey 
may  prevent  or  reduce  the  learning.  (2)  Frequ  kni  KR  or  instant  KK  can 
reduce  the  learning  of  error-detection  capability  ^eca  lse  there  is  no 
time  for  learners  to  think  about  feedback. 

One  way  to  overcome  the  possible  negative  erects  of  Join;  K  K 
feedback  is  to  provide  visual  messages  that  represent  the  Englisn  'is 
verbatim  text,  along  with  repealing  the  audio.  By  designing  the 
instruction  with  short  passages,  phrases  and  sentences  the  length  oi 
t  edback  remains  :>hort  Captioning  's  a  good  (example  of  the  exact 
order  of  verbatim  thai  reproduces  the  words  exacti)  *b  they  are  <*n 
in  the  movicr  r>r  in  tele  ision  )  u grains,   because  of  thr  \\v Uod 
capacity  of  shovMerm  mcuiory,  Hie  captioning  i?  4 his  study  v\.<s 
design  "d  as  a  short  sentence. 

In  this  study,  captioning  w-s  designs  d  as  a  urncm,  i  oi 
knowledge  of  results  feeobark  (KK)  v  '    ii  servo1  as    forma ti  »n  in  Uu> 
learner  thai  enhances  goal  alignment  C  L  n,  1987;       *mmij,  'a  umidt, 
ik  Walter,  1984;  Swim  an,  Schmidt,  N!r,hol|,.on/  Jt  Shap>o  (1990). 
Learners  were  encouraged  fo  acbwly  in  "rpivi  cupHo  ing  :n  or<Jer  to 
improve  listening  eomprohens'  >r    Very  I  '  U  invesljgai ><>%  has  been 
conducted  on  the  eifec'  of  captioning  ^  knovvk'dr  of  resulv  #,«\H>nek. 

Si'ATiiMi .i  T  OW  H  IK  Pmiiu-M 

The  pii7iA.".e  (/  this  ;  .is/  '  was  to    wsli'Mip  iiv  n(("(  (  oi 
caotioiuny  as  know1MHjje  of  res**,    :"'vdbai "  .  »u  \v  l<  nut* 
comprehension  of  Hi^l  >h  as  a  :,eu,rd  ?ang.      1  in  ;     mpu"  i  .  is< 
;,:rrr;K;iv(  videodisc  syst^n.  "lie  res*  mi     uuesfi  ni     v««  lig,  ifu  ^ 
this  f-!i!iiy  were; 

1,   Whr«<  \      <:he  c«f    t  O'   -  'lionn-si-  ir.^O   *    »iowi<  »,  e  ot 
recalls  feedback  op  usii/iung  o,;..>r  .^ni-ai.    of  ivs>  \l  l\      <i  f<unu 
language  d.s  mea;i:,vf o\     hsu'itin^    - ; f/i t im  '  "ion        ^   ,.;  'he 
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movie  The  Last  Emperor  and  the  TOEFL  Listening  Comprehension 
test? 

2.  Did  above  average  reading  proficiency  students  have  higher 
achievement  on  a  listening  comprehension  test  using  the  movie  The 
Last  Emperor  and  the  TOEFL  Listening  Comprehension  test  than  below 
average  English  reading  proficiency  students? 

3.  Was  there  a  statistically  significant  interaction  between 
treatment  and  English  reading  proficiency  for  performance  on  a 
listening  comprehension  test  ^>ng  the  movie  The  Last  Emperor  and 
the  TOEFL  Listening  Comprehend  n  test;' 

METHOD 

Sample 

Seventy-two  freshmen  at  National  Taiwan  Normal  University/ 
Taipei,  Taiwan,  the  Republic  of  China,  sixty-one  female  and  elev  n 
male,  participated  in  this  study.  The  average  age  was  19  years.  All 
subjects  were  enrolled  in  the  mandatory  Freshmen  English  cours* 
The  freshmen  population  was  1035.  Of  this  population,  all  English 
majors  and  industrial  education  majors  were  excluded  leaving  955 
freshmen.  Industrial  education  majors  were  not  involved  in  the  study 
because  they  did  not  take  the  Joint  College  Entranced  Examination. 
Because  captioning  has  a  close  relationship  With  reading  ability,  the 
subjects  of  this  study  were  assigned  to  above  or  below  reading 
proficiency  groups  according  to  their  English  scores  on  the  Joint  Colleg< 
Entrance  Examination.  The  mean  and  stands  -d  deviation  for  English 
Achievement  on  the  Joint  College  Entrance  Examination  were  58.uft 
and  13.14  respectively.  Tnis  left  296  students  abov<  and  296  below. 
Thirty-six  above  and  thirty-six  below  average  English  reading 
proficiency  students  were  randomly  selected  and  randomly  is«,igned  u 
onr  of  two  treatment  groups  so  that  eighteen  high  .ind  eighteen  low 
subjects  were  in  each  treatment  grc  ip. 

Materials 

Two  versions  oi  an  interactive  videodisc  j    --grain  designed 
around  the  movie  The  j  as  t  Emperor  were  developed    'W  version, 
the    ptioning  treatment,  used  caption  /sg  ir  knowledge  oi  results 
feedback.  The  second  version,  the  no  caption* j  ;  treatment,  received 
no  captioning  us  p-'-rt  of  the  feedback.  There  were  no  other  differences 
between  versions.  The  Last  Emperor  »  as  chosen  in  part  because  it 
contains  both  ''on tent  and  bountiful  dialogue  tu..'  fan  be  used  for 
irarmiv  vocabulary,  idiomatic  expressions,  and  language 
comprehension.  Most  oi  the  students  km  ■•'  the  general  history  of  ';,he 
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Last  Emperor,  providing  some  relevance.  In  addition,  the  movie 
included  anecdotes  used  to  increase  the  entertainment  effect  which 
were  not  familiar  to  the  students. 

Lesson  Description 

Once  the  students  booted  the  computer  and  initialized  the 
videodisc  player,  a  title  screen  was  presented,  then  the  system  would 
ask  the  students  to  type  in  their  student  number.  The  system  then 
confirmed  whether  he/she  was  a  subject  and  in  which  treatment  group 
she/he  belonged.  If  he/she  was  not  a  subject  then  the  system  would 
exit.  If  he/she  was  a  subject  then  the  objectives  and  directions  would 
be  presented  in  English. 

In  the  captioning  treatment  group,  students  watched  a  complete 
chapter  of  The  Last  Emperor.  Then  they  viewed  a  series  of  clips, 
answering  embedded  questions  about  each  clip.  The  embedded 
questions  in  the  chapter  were  multiple-choice  questions,  designed 
using  the  same  format  as  the  Listening  Comprehension  subtest  of 
TOEFL.  (The  TOFEL  format  consists  of  spoken  statements,  short 
conversations,  and  several  short  talks  or  presentations.)  Students  were 
encouraged  to  comprehend  the  ideas,  the  gist,  rather  than  every  word. 
Students  were  asked  to  answer  using  the  contextual  cues. 

If  they  answered  correctly,  they  reviewed  the  clip  again  with 
captioning  used  as  a  knowledge  of  results  feedback  to  confirm  every 
correct  response.  The  program  then  branched  the  students  to  the  next 
clip.  If  they  answered  incorrectly,  they  reviewed  the  clip  one  more 
time  and  were  asked  to  answer  the  question  again.  If  they  still 
answered  incorrectly,  the  captioning  was  displayed  on  the  monitor, 
revealing  the  correct  answer.  Captioning  served  as  information  for  the 
learners  to  interpret.  Learners  had  only  two  tries  per  question  and  then 
they  continued  to  the  next  clip.  After  completing  each  chapter, 
students  received  a  summary  display  concerning  their  performance. 
The  system  then  displayed  the  menu  indicating  the  previous 
chapter(s),  the  next  chapter,  and  quit  option.  Only  those  students  who 
were  90%  correct  on  the  first  try  coul  i  continue  with  the  next  chapter. 

In  the  no  captioning  treatment  group,  students  received  the 
same  videodisc-based  program  without  captioning  in  the  feedback.  If 
they  answered  a  question  correctly,  they  received  the  positive  feedback, 
but  without  captions.  If  they  answered  incorrec'ly,  they  received 
xvegative  feedback,  and  viewed  that  clip  one  more  time  and  were  asked 
to  answer  the  question  again.  If  they  still  answered  incorrectly,  the 
nswor  was  displayed  on  the  monitor  without  captioning.  There  were 
in  other  differences  between  the  treatments. 

The  criterion  of  90%  correct  on  the  first  try  was  selected  to  ensure 
hat  students  had  masteied  the  material  before  proceeding  to  the  next 
uiapter.  Bloom  (1976)  a'gues  that  mastery  learning  can  be  used  as  a 
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way  of  eliminating  individual  differences  among  students  in  their 
ability  to  learn.  Hence,  in  this  study,  aU  of  the  subjects  would  have  the 
same  mastery  level  before  they  took  the  post-test.  If  there  was  a 
significant  difference  between  the  groups,  it  was  because  of  the 
treatment  effect  or  the  different  reading  proficiency  level. 

Procedure 

Prior  to  icceiving  the  treatments,  all  subjects  were  given  the 
TOEFL  Listening  Comprehension  subtest,  used  as  a  pretest.  Then  the 
two  treatment  groups  used  The  Last  Emperor  one  hour  a  week  in 
alternating  weeks  for  ten  weeks.  The  captioning  group  practiced  the 
first  week  and  the  no  captioning  group  followed  the  second  week. 
Most  subjects  were  able  to  practice  the  entire  program  in  the  5  hours' 
time  limit.  Some  subjects  had  to  repeat  the  same  chapter  if  they  were 
not  90%  correct  on  the  first  try.  At  the  end  of  twelve  weeks,  all  subjects 
took  the  post-tests:  the  movie  test  and  a  second,  equivalent  form  of  the 
TOEFL  Listening  Comprehension  subtest. 

Instruments 

The  instruments  used  in  this  study  were  a  listening  test  based  on 
the  movie  The  Last  Emperor  and  two  equivalent  forms  of  the  TOEFL 
Listening  Comprehension  test  of  the  Educational  Testing  Service. 
They  were  admi:  tistered  to  investigate  the  knowledge  of  results  effect 
of  captioning  on  above  or  below  average  English  reading  proficiency 
using  a  computer-based  interactive  videodisc  system. 

Validity  and  Reliability  of  the  Movie  Test 

The  test  based  on  the  movie  The  Last  Emperor  was  constructed 
using  the  same  format  and  with  the  same  number  of  questions  as  the 
TOEFL  Listening  Comprehension  subtest:  fifty  multiple-choice 
questions  administered  in  thirty-five  minutes.  In  addition,  none  of  the 
questions  in  the  movie  test  were  the  same  as  the  embedded  questions 
in  the  treatments.  The  test  was  recorded  from  the  videodisc  program 
of  The  Last  Emperor  using  audio  only,  and  consisted  of  spoken 
statements,  short  conversations,  and  several  short  talks. 

In  order  to  decide  if  the  test  items  were  appropriate,  two 
American  graduate  students  and  one  Chinese  professor  who  teaches 
listening  comprehension  of  English  as  a  second  language  conducted  an 
item  analysis  of  the  movie  test.  The  following  Likert-type  scale  was 
used:  (1)  Very  Inappropriate,  (2)  Inappropriate,  (3)  Undecided,  (4) 
Appropriate,  and  (5)  Very  Appropriate.  Only  test  items  scored  on  or 
above  the  Appropriate  level  by  all  three  raters  were  used.  A  split-half 
reliability  of  the  movie  test  was  conducted,  using  the  Spearman-Brown 
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formula  to  estimate  the  reliability  of  the  whole  test.  The  split-half 
reliability  coefficient  of  the  movie  test  was  .71. 

Statistical  Design 

This  study  was  a  pretest-posttest  control  group  design  (Campbell 
&  Stanley,  1963).  The  dependant  variable,  listening  comprehension, 
was  analyzed  with  a  2x2  (treatment  x  reading  proficiency)  analysr  of 
covariance  (ANCOVA)  to  test  the  following  research  hypotheses: 

1.  Students  in  the  captioning  as  knowledge  of  results  feedback 
treatment  have  significantly  higher  posttest  mean  scores  on  the  movie 
test  than  students  in  the  no  captioning  treatment. 

2.  Above  average  English  reading  proficiency  students  have 
significantly  higher  posttest  mean  scores  on  the  movie  test  than  below 
average  English  reading  proficiency  students  after  completing  the 
treatments. 

3.  There  is  a  significant  interaction  between  treatment  and 
reading  proficiency  for  pos;test  mean  scores  on  the  test  of  movie  The 
Last  Emperor. 

4.  Students  in  the  captioning  as  knowledge  of  results  feedback 
treatment  have  significantly  higher  posttest  mean  scores  on  the  TOEFL 
Listening  Comprehension  subtest  than  students  in  the  no  captioning 
treatment. 

5.  Above  average  English  reading  proficiency  students  have 
statistically  higher  posttest  mean  scores  on  the  TOEFL  Listening 
Comprehension  subtest  than  below  average  English  reading 
proficiency  students  after  completing  the  treatments. 

6.  There  is  a  significant  interaction  between  treatment  and 
reading  proficiency  for  posttest  mean  scores  on  the  TOEFL  Listening 
Comprehension  subtest. 

Analysis  of  covariance  reduces  the  effects  of  initial  group 
differences  statistically  by  making  adjustments  to  post-test  scores.  The 
TOEFL  pretest  score  was  used  as  the  covariate  to  adjust  for  both  post- 
test  scores. 

Results 

Each  individual  used  the  interactive  videodisc  progran  for  five 
hours  over  ten  weeks.  The  program  consisted  of  seven  chapters. 
Chapters  one,  two,  three,  and  six  took  an  average  of  50  minutes  to 
complete.  Chapters  four,  five,  and  seven  took  an  average  of  30 
minutes  to  complete.  Because  of  the  subjects'  tight  class  schedules,  the 
study  was  conducted  in  the  evening  and  at  noon,  even  on  Saturday 
and  Sunday.  All  subjects  completed  the  first  six  chapters.  Two  subjects 
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of  below  average  reading  proficiency  in  the  captioning  group 
completed  all  of  seven  chapters. 

All  hypotheses  were  tested  at  the  a  =  0.05  level  of  significance. 
The  analysis  of  covariance  (ANCOVA)  was  employed  to  evaluate  the 
strength  of  the  treatment  based  on  the  movie  test  of  The  Last  Emperor 
and  listening  comprehension  scores  of  TOEFL,  using  the  pre-test 
performance  scores  of  the  TOEFL  Listening  Comprehension  test  as  the 
covariate.  The  results  of  the  pretest  and  both  posttests  are  shown  in 
Table  1. 


Insert  Table  1  about  here 


Hypotheses  Testing  Related  to  the  Movie  Test 

The  first  analysis  of  covariance  was  used  to  determine  the  effects 
of  the  treatment  and  reading  proficiency  on  performance  on  the  movie 
test  of  The  Last  Emperor.  The  dependent  variable,  listening 
comprehension,  was  analyzed  with  a  2x2  (Treatment  x  Level  of 
Reading  Proficiency)  analysis  of  covariance.  The  pretest  of  the  movie 
test  was  used  as  covariate  to  adjust  for  post-lest  score. 

The  interaction  of  treatment  and  reading  proficiency  was 
analyzed  to  determine  if  high  or  low  reading  proficiency  students 
differed  with  or  without  captioning  on  the  movie  test.  As  shown  in 
Table  2,  there  was  no  interaction  between  treatment  and  reading 
proficiency,  F(l,  67)  =  2.16,  p_  =  .15  (>  .05). 


Insert  Table  2  about  here 


Since  there  was  no  interaction,  the  main  effects  of  treatment  and 
reading  proficiency  were  examined.  Table  2  indicates  that  there  was  a 
statistically  significant  difference  between  groups  due  to  the  treatment, 
F,(l,  67)  =  144.21,  ^  <  .01  (  a  <  .05).  Subjects  who  received  the  captioning 
treatment  had  higher  posttest  mean  scores  on  the  movie  test. 

Table  2  also  shows  that  there  was  a  statistically  significant 
difference  between  the  above  average  English  reading  proficiency 
group  and  the  below  average  English  reading  proficiency  group  on  the 
movie  test,  F(l,  67)  =  34.25,  g  <  .01  (  a  <  .05).  Students  in  the  above 
average  reading  proficiency  group  performed  significantly  better  on  the 
movie  test. 

Hypotheses  Testing  Related  to  the  TOLFL  Listening  Comprehension 
Test 
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The  second  analysis  of  covariance  was  used  to  measure  the 
effects  of  the  treatment  and  reading  proficiency  on  performance  on  the 
TOEFL  Listening  Comprehension  test.  The  dependent  variable, 
listening  comprehension,  was  analyzed  with  a  2x2  (Treatment  x  Level 
of  Reading  Proficiency)  analysis  of  covariance.  TOEFL  pre-test  was  used 
as  covariate  to  adjust  for  post-test  score. 

Insert  Table  3  about  here 


As  in  the  case  of  the  movie  test,  Table  3  shows  that  there  was  no 
interaction  between  treatment  and  reading  proficiency,  F(l,  67)  =  1.43,  p. 
=  .24  (  >  .05).  Table  3  also  indicates  that  the  main  effect  of  the  treatment 
was  not  statistically  significant,  F(l,  67)  =  0.68,  j)  =  .41  (  a  >  .05).  Subjects 
who  received  the  captioning  treatment  had  no  statistically  higher  post- 
test  mean  scores  on  the  TOEFL  Listening  Comprehension  test  than 
students  who  received  the  no  captioning  treatment. 

However,  Table  3  does  show  a  statistically  significant  difference 
between  the  above  average  English  reading  proficiency  group  and  the 
below  average  English  reading  proficiency  group  on  the  TOEFL 
Listening  Comprehension  subtest,  £(1,  67)  =  26.72,  p  <  .01  (  a  <  .05).  The 
above  average  reading  proficiency  group  performed  better  than  the 
below  average  reading  proficiency  group. 

Summary  of  the  Findings 

Statistical  analysis  of  the  data  from  this  study  led  to  the 
following  findings: 

1.  Based  on  the  analysis  of  covariance,  there  was  a  statistically 
significant  difference  between  the  treatment  groups  on  scores  on  the 
movie  test,  such  that  the  captioning  group  performed  better  than  the 
no  captioning  group. 

2.  The  analysis  of  covariance  also  showed  that  there  was  a 
statistically  significant  difference  between  reading  proficiency  groups  in 
post-test  mean  scores  on  the  movie  test  in  favor  of  the  above  average 
reading  proficiency  group. 

3.  The  analysis  of  covariance  revealed  that  there  was  no 
interaction  between  treatment  and  reading  proficiency  in  post-test 
mean  scores  on  the  movie  test. 

4.  The  analysis  of  covariance  indicated  that  there  was  no 
statistically  significant  difference  between  the  treatment  groups  mean 
scores  on  the  TOEFL  Listening  Comprehension  test. 

5.  The  analysis  of  covariance  showed  that  there  was  a 
statistically  significant  difference  between  reading  proficiency  groups  in 
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post- test  mean  scores  on  the  TOEFL  Listening  Comprehension  test,  in 
favor  of  the  above  average  reading  proficiency  group. 

6.  The  analysis  of  covariance  revealed  that  there  was  no 
interaction  between  treatment  and  reading  proficiency  in  post-test 
mean  scores  on  the  TOEFL  Listening  Comprehension  test. 

DISCUSSION 

The  results  of  thii?  study  indicate  that  there  is  no  interaction 
between  treatment  and  reading  proficiency  for  post-test  mean  scores  on 
the  movie  test  and  the  TOEFL  Listening  Comprehension  subtest. 
Further,  they  show  that  students  who  receive  captioning  as  knowledge 
of  results  feedback  on  the  interactive  videodisc  program  have 
statistically  higher  post-test  mean  scores  than  students  who  receive  no 
captioning  on  the  movie  test.  This  finding  is  consistent  with  the 
findings  of  Boyd  and  Vader  (1972)  and  Murphy-Berman  and  Jorgensen 
(1980). 

Therefore,  it  seems  that  captioning  is  effective  on  listening 
comprehension  of  the  specific  subject  matter  presented.  This  .esult  is 
inconsistent  with  the  findings  of  Branvold,  Chang,  Probst,  and 
Bennion  (1986),  who  investigated  the  effectiveness  of  an  interactive 
videodisc  workstation  in  teaching  content,  vocabulary,  and  idiomatic 
expressions  using  the  movie  Raiders  of  the  Lost  Ark.  Significant 
differences  in  effectiveness  between  workstations  were  found  only  in 
the  area  of  idiomatic  expressions.  The  current  study  does  not 
specifically  address  idiomatic  understanding,  though  idiomatic 
expressions  are  very  important  for  understanding  content.  However, 
this  study  does  find  an  effect  on  specific  content  comprehension. 

The  results  also  show  that  students  in  the  above  average  reading 
proficiency  group  have  higher  posttest  mean  scores  than  the  below 
average  reading  proficiency  group  on  the  movie  test  and  on  the  TOEFL 
Listening  Comprehension  test.  Therefore,  listening  comprehension 
has  a  close  relationship  with  reading  proficiency.  This  finding  is  again 
congruous  with  the  findings  of  Murphy-Berman  and  Jorgensen  (1980). 
It  is  suggested  that  students  trying  to  improve  their  English  listening 
comprehension  skills  should  learn  more  about  written  English 
vocabulary  and  idiomatic  expressions,  as  well  as  auditory  intonation 
and  stress  of  vocabulary.  Additionally,  students  need  to  learn  more 
about  slang  or  common  expressions  that  are  usually  heard  in 
television  programs,  in  the  movies,  and.  in  daily  life. 

During  the  study,  informal  observation  revealed  that  subjects 
used  background  knowledge  in  listening  comprehension.  They  knew 
the  history  and  the  main  characters  of  the  movie  The  Last  Emperor. 
This  observation  is  consistent  with  the  findings  of  Johnson  11981), 
Kitao  (1989),  Long  (1989,  1990),  Samuels  (1984),  and  Van  Dijk  (1980). 
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This  use  of  background  knowledge  is  not  likely  to  be  a  confounding 
variable  since  subjects  were  randomly  assigned  to  groups  and  the 
content  was  the  same  for  all  subjects.  Background  knowledge  has  no 
effect  on  the  TOEFL  Listening  Comprehension  test,  because  TOEFL  test 
is  developed  for  evaluating  students'  general  English  proficiency. 

Schema  or  cultural  background  knowledge  is  an  essential 
component  of  second  language  comprehension.  However,  Kitao  (1989) 
argues  that  second  language  classes  generally  use  the  grammar- 
translation  method,  which  does  not  emphasize  the  use  of  background 
knowledge.  This  is  true  in  Taiwan.  Previous  studies  indicate  that 
schemata  can  be  taught  (Brooks  &  Dansereau,  1983;  Brown,  Smiley, 
Day,  Townsend,  &  Lawton,  1977).  Hence,  second  language  educators 
need  to  know  which  instructional  designs  and  strategies  for  teaching 
scripts  (schema)  will  be  most  effective  and  efficient  in  specific 
instructional  environments.  It  is  possiMe  that  background  knowledge 
helped  students  understand  the  contc...  .md  context  of  the  movie  The 
Last  Emperor.  The  effect  of  backgrc  und  knowledge  remains  to  be 
investigated. 

In  this  study,  captioning  es  KR  feedback  was  provided  under  two 
conditions  in  the  captioning  treatment  group.  When  a  subject 
answered  correctly,  captioning  was  provided  after  a  delay  of  three 
seconds.  When  a  subject  ansv.  ered  question  incorrectly  twice, 
captioning  was  also  provided  after  a  delay  of  three  seconds.  Hence,  the 
captioning  feedback  may  be  regarded  as  delay  of  knowledge  of  results 
feedback.  Swinnen  et  al.  (1990)  investigated  motor  skills  learning 
under  three  conditions:  (1)  an  instantaneous  KR  condition  in  which 
subjects  received  KR  immediately  after  response  completion,  (2)  a 
delayed  KR  condition  in  which  subjects  had  a  KR-delay  interval  of 
eight  seconds,  and  (3)  an  estimation  condition  in  which  subjects  were 
required  to  estimate  their  movement  time  during  an  eight  seconds  KR- 
delay  interval.  They  report  that  because  instantaneous  KR  provides 
the  learners  with  information  about  the  success  of  their  response,  it 
reduces  the  subjects'  motivation  to  evaluate  their  responses  as 
frequently  or  intensively  as  subjects  in  the  delayed  or  estimation 
conditions.  Thus,  "instantaneous  KR  can  discourage  the  processing  of 
other  kinds  of  information,  such  as  intrinsic  response-produced 
feedback  that  would  lead  to  the  learning  of  the  capability  to  detect 
errors  in  future  performances"  (p.  715).  They  conclude  that 
instantaneous  KR  degrades  learning  because  it  reduces  the 
development  of  error- detection  capabilities. 

Although  the  Swinnen  et  al.  (1990)  study  was  not  conducted 
using  ini  tractive  video,  any  one  of  the  three  KR  conditions  could  be 
applied  to  the  design  of  an  interactive  videodisc  program.  During  the 
current  study,  it  was  informally  observed  that  subjects  were  actively 
examining  the  captioning  in  order  to  find  out  what  they 
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misunderstood  when  they  answered  incorrectly.  This  learning  process 
may  lead  to  learning  error-detection  capabilities  and  actively  involves 
the  student  in  caption  interpretation.  Subjects  took  notes,  consulted  an 
English  dictionary  or  idiomatic  handbook,  or  discussed  the  results  with 
the  investigator.  These  observations  provide  evidence  that  knowledge 
of  results  feedback  has  a  strong  informational  property  that  encourages 
subjects  to  use  this  information  for  performance  enhancement.  KR 
also  allows  subjects  to  feel  comfortable  with  their  mastery  of  the 
material.  These  findings  are  consistent  with  the  findings  of  Swinnen 
et  al.  (1990)  and  Cohen  (1985). 

During  the  experiment,  subjects  were  observed  operating  the 
interactive  videodisc  system,  and  informally  interviewed.  Subjects 
indicated  that  the  interactive  videodisc  program  was  highly 
motivational  and  interesting.  This  observation  is  consistent  with  the 
finding  of  Branvold  et  al.  (1986),  Koskinen,  Wilson,  &  Jensema  (1986), 
and  Kearsley  (1990).  Some  subjects  added  that  the  material  was  easy  to 
understand  because  of  the  contextual  cues,  body  language,  and  facial 
expressions  of  the  speakers.  However,  one  cannot  be  certain  that  the 
students'  high  motivation  is  not  due  to  the  use  of  the  new  technology 
of  interactive  video.  Hence,  the  motivation  issue  must  be  investigated. 
Such  investigation  requires  longer  treatments,  experienced  users  of  the 
technology,  and  a  measuring  instrument  for  motivational  effect. 

An  alternative  approach  to  investigate  the  motivation  issue 
would  be  the  use  of  Keller's  ARCS  model  (1987)  as  the  basis  for 
analyzing  the  motivational  design  of  the  instructional  treatment. 
ARCS  is  an  acronym  that  stands  for  attention,  relevance,  confidence, 
and  satisfaction.  With  respect  to  attention,  the  movie  The  Last 
Emperor  has  won  many  Oscar  Awards  and  tells  a  fascinating  story.  It 
gains  and  maintains  student  attention.  With  respect  to  relevance, 
Taiwanese  students  are  familiar  with  the  content  of  The  Last  Emperor 
and  it  relates  to  their  modern  Chinese  history  class.  Students  may  be 
motivated  to  compare  what  they  see  in  the  movie  with  what  they 
study  in  their  textbook.  W'jh  respect  to  confidence,  in  this  program, 
90%  correct  on  the  first  try  of  answering  the  embedded  questions  was 
necessary  to  move  to  the  next  chapter.  Learners  are  aware  of 
evaluative  criteria  and  performance  requirements  (Keller  &  Kopp, 
1987).  In  addition  to  stimulating  the  need  to  achieve,  it  also  helps  build 
an  expectancy  that  success  will  be  due  to  personal  effort  if  the  goal  is 
achieved  (Keller  &  Kopp,  1987).  With  respect  to  satisfaction,  KR  allows 
subjects  to  feel  comfortable  with  their  mastery  of  the  material.  In 
addition,  "The  knowledge  of  results  promotes  learning  by  confirming 
correct  responses  and  correcting  wrong  ones,  but  it  also  has  a 
motivational  effect  in  that  it  allows  the  achievement-motivated  person 
to  get  a  measure  of  success"  (Keller  &  Kopp,  1987,  p.  315). 
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The  treatment  used  in  this  study  had  no  apparent  impact  on 
general  listening  comprehension  skills.  However,  this  may  be  due  to 
the  use  of  only  one  videodisc  program  and  limited  practice.  It  is 
difficult  to  master  listening  comprehension  of  a  second  language  with 
only  five  hours  practice  over  ten  weeks  on  only  one  content  area.  It  is 
possible  that  an  effect  on  the  general  listening  skills  measured  by  the 
TOEFL  Listening  Comprehension  test  could  be  achieved  if  more 
videodisc  programs  on  a  wide  variety  of  topics  were  used  and  if  much 
more  practice  time  was  provided  for  the  students. 

Conclusions  and  Implications 

In  consideration  of  the  limitations  in  this  study  and  the 
statistical  analysis  of  the  data,  the  following  conclusions  were  drawn: 

1.  Based  on  the  finding  that  significant  differences  existed 
between  the  treatment  groups  on  the  movie  test,  it  was  concluded  that 
captioning  used  as  knowledge  of  results  feedback  improves  listening 
comprehension  on  specific  content  for  Taiwanese  college  freshmen 
students. 

2.  Based  on  the  finding  that  no  significant  differences  existed 
between  the  treatment  groups  on  the  TOEFL  Listening  Comprehension 
subtest,  it  was  concluded  that  the  captioning  used  as  knowledge  of 
results  feedback  does  not  improve  general  listening  comprehension 
skills.  However,  this  result  may  be  due  to  limited  exposure  to  the 
treatment:  five  hours  over  ten  weeks.  It  may  also  be  due  to  the  lack  of 
content  variety. 

3.  Based  on  the  finding  that  significant  differences  also  existed 
between  the  above  average  English  reading  proficiency  group  and  the 
below  average  English  reading  proficiency  group  on  the  movie  test  and 
the  TOEFL  Listening  Comprehension  subtest,  it  was  concluded  that 
English  listening  comprehension  may  be  closely  related  to  English 
reading  proficiency. 

4.  Informal  interviews  with  some  subjects  during  the  study 
indicated  that  they  were  not  embarrassed  or  intimidated  when  they 
could  not  understand  the  content  of  the  videodisc  program.  They 
stated  that  the  learning  experience  was  pleasant  and  free  from  anxiety. 
From  these  comments  it  may  be  concluded  that  the  videodisc  and 
microcomputer  technologies  have  demonstrated  a  friendly 
environment  that  can  be  easily  integrated  into  teaching  foreign 
languages  and  reduce  students'  anxiety  and  embarrassment. 

The  present  results  have  implications  for  educational  practice. 
Informal  observations  suggest  that  captioning  as  KR  may  encourage 
the  learners  to  examine  new  vocabulary,  idiomatic  expressions,  and 
'  their  usage.  Captioning  as  knowledge  of  results  feedback  does  appear  to 
encourage  learners  to  actively  interpret  the  information,  draws 
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learners  attention,  and  facilitates  the  learning  of  a  second  language. 
However,  the  findings  show  that  the  specific  content  treatment  did  not 
improve  general  listening  comprehension  skills.  If  the  goal  is  to 
improve  general  listening  comprehension,  then  treatments  of 
covering  many  different  subject  areas  may  be  used.  If  the  goal  is  to 
improve  comprehension  of  specific  content  then  captioning  provides  a 
tool  to  assist  the  learner. 

The  current  findings  and  students'  reactions  suggest  that  an 
interactive  videodisc  system  has  the  potential  to  improve  English  as  a 
second  language  learning.  This  is  especially  important  since  qualified 
English  language  teachers  for  listening  and  speaking  skills  are  in  short 
supply  in  Taiwan  and  learners  differ  widely  in  entry  skills. 
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Table  1 

Pre-test  and  Post-test  for  the  TOEFL  Listening  Comprehension  and  the 
Movie  Test 


Treatment 
Reading 

Proficiency  Captioning  No 

Captioning 


n 

M 

SD 

M 

SD 

Above 

TOEFL  Pre-test 

36 

42.94 

4.32 

41.67 

3.55 

Average 

TOEFL  Post-test 

36 

46.56 

3.90 

45.67 

2.99 

Movie  Post-test 

36 

66.72 

1.36 

60.39 

2.30 

Below 

TOEFL  Pre-test 

36 

39.17 

2.50 

39.94 

3.70 

Average 

TOEFL  Post-test 

36 

41.44 

3.22 

41.17 

3.84 

Movie  Post-test 

36 

62.83 

2.43 

55.50 

3.82 

Table  2 

Two-Way  Analysis  of  Covariance  for  the  Movie  Test  by  Treatment  and 
Reading  Proficiency 

Source  SS1  df  MS^  F  p 

_____  263.85  1  263T8~5  46.37 

Treatment  (T)  820.58  1  820.58  144.21*  <  .01 

Reading  Proficiency  (R)  194.90  1  194.90  34.25*  <  .01 

TxR  12.28  1  12.28  2.16  .15 

Within  381.52  67  5.69 

Total  1662.61  71 


*  95Fl,67  =  3.99,  a  <  .05. 

Note:  "The  SS  do  not  normally  add  up  to  the  model  SS."  (SAS/STAT 
user's  guide,  1990,  p.  936). 
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Table  3 

Two-Way  Analysis  of  Covariance  for  the  TOEFL  Listening 
Comprehension  Test  by  Treatment  and  Reading  Proficiency 


Source  SS'         df  MS'         F  p 


Covariates 

900.85 

1 

900.85 

2.39.50 

Treatment  (T) 

2.56 

1 

2.56 

.68 

Reading  Proficiency  (R) 

100.49 

1 

100.49 

26.72*  < 

TxR 

5.37 

1 

5.37 

1.43 

Within 

252.01 

67 

3.76 

Total 

1261.28 

71 

*.95Fi,67  =  3.99,  a  <  .05. 
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Introduction 


Faculty  development  is  an  issue  at  most  institutions  of  higher  education 
(Alfonsi,1990).  An  increasing  knowledge  base  in  most  disciplines,  a  more  culturally  diverse 
student  body,  more  stringent  retention,  tenure  and  promotion  requirements,  an  increased 
interest  in  faculty  evaluation,  and  a  demand  for  alternative  modes  of  instruction  have  fostered 
and  even  demanded  interest  in  faculty  development  programs  (see  Diehl  and  Simpson,  1989; 
Berman  and  Skeff,  1989;  American  Association  of  University  Profoessors,  1990).  Alternative 
modes  of  instruction  have  begun  inroads  on  the  traditional  method  of  content  delivery-the 
lecture.  These  modes  of  instruction  often  emphasize  instructional  technology -both  in  design 
and  planning  and  in  delivery  (University  of  Maryland,  1984). 

Faculty,  too  often,  are  poorly  trained  in  educational  methods  and  the  newer  technologies 
of  instruction.  Instructional  technologists  by  their  nature  and  training  are  often  proactive  and 
tend  to  take  a  lead  in  programs  aimed  at  change-especially  in  the  areas  of  instruction.  Many 
workshop  and  seminar  offerings  in  faculty  development  (FD)  programs  are  presented  by 
instructional  technologists  and  include  content  related  to  instructional  technology  (Marbler  and 
Lawyer,  1984;    Albright,  1989). 

Purpose  of  Study 

The  purpose  of  this  study  is  to  investigate  the  role  of  instructional  technology  (IT) 
related  to  planning,  delivery,  evaluation  and  content  selection.  We  want  to  investigate  what 
internal  organizations  of  the  university  sponsor  and  deliver  FD  offerings  and  where  the 
ultimate  responsibility  of  the  FD  program  lies.  We  wish  to  determine  the  types  of  offerings 
related  to  IT  and  their  relative  frequency  and  attendance.  Answers  to  questions  related  to 
planning,  evaluation,  and  effect  on  instruction  and  faculty  growth  and  involvement  are  sought. 

Methodology  of  Study 

Two  types  of  questionnaires  were  designed  and  used  in  this  study.  Three  types  of 
respondents  were  surveyed.  The  sample  was  the  170  USA  colleges  and  universities  included  in 
Masters  Curricula  in  Educational  Communications  and  Technology:  A  Descriptive 
Directory  by  Johnson,  et  al,  1989.  Each  of  these  institutions  was  identified  as  having  either 
MA  or  Ph.D.  programs  in  educational  communications  and  technology.  Since  the  major  emphasis 
of  this  Faculty  Development  study  was  related  to  instructional  technology  it  seemed  logical  that 
having  an  academic  program  in  educational  communications  and  technology  would  have  some 
effect  on  the  planning,  development,  and  evaluation  of  such  programs  and  on  the  curricular 
content  of  the  FD  program. 

The  subjects  participating  in  this  study  were  media  services  directors,  department 
heads  of  instructional  technology  academic  programs  and  the  person  in  the  institution 
responsible  for  the  faculty  development  program  (FD  officer).  Three  color-coded 
questionnaires  were  sent  to  the  chief  academic  officer  who  was  requested  to  see  that  the 
appropriate  persons  received  them.  The  questionnaires  directed  to  the  media  services  directors 
and  IT  academic  head  were  the  same  and  requested  the  perceptions  of  these  subjects  concerning 
eleven  statements  relevant  to  the  FD  program  at  their  institution.  The  questionnaire  directed  to 
the  institution's  FD  officer  included  these  same  items,  but  additional  data  was  requested  related 
to  campus  organizations  and  departments  which  offered  FD  workshops/seminars  and 
administrative  responsibility,  types  of  workshops/seminars  offered  with  instructional 
technology  content  and  programmatic  planning. 
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The  eleven  statements  to  which  the  respondents  reacted  related  to  planning,  evaluation, 
input,  support,  impact,  promotional  activities,  resources  and  the  role  of  instructional 
technologists  in  faculty  development.  A  semantic  differential  was  used  which  included  Strongly 
Agree,  Agree,  Agree  Slightly,  Disagree  Slightly,  Disagree*;  and  Strongly  Disagree.  The  responses 
of  these  three  types  of  subjects  were  statistically  compared  by  computing  the  Average  Weighed 
Value  and  then  using  a  Rank  Difference  Correlation  (Spearman  rho). 

A  Previous  Study 

Some  data  from  a  similar  study  (Stephens,  1991),  reported  at  the  1991  Association 
for  Educational  Communications  and  Technology  conference  in  Orlando,  are  incorporated  into 
this  study.  The  present  study  was  done  as  a  follow-up  using  the  same  institutions  surveyed  in 
1990.  In  the  first  study,  a  survey  instrument  was  sent  to  the  person  listed  as  being  the  chair 
of  the  program  offering  masters  or  doctoral  degrees  in  educational  communications  and 
technology.  One  hundred  and  twenty-four  institutions  responded.  In  the  latter  study,  the 
number  of  institutions  responding  is  an  unknown,  because  three  different  types  of  respondents 
were  surveyed--the  FD  officer,  the  director  of  educational  media  services,  and  the  academic 
head  of  the  IT  program.  At  least  65  institutions  responded,  but  the  actual  number  is  probably 
more  as  no  institution  identifying  code  was  used.  A  color  code,  however,  was  used  to  delineate 
the  type  of  respondent.  Sixty-five  FD  officers,  56  directors  of  media  service  centers  (IRC), 
and  47  heads  of  IT  academic  programs  responded. 

Findings 

Administrative  RespnnRihj|jt,y 

In  the  1990  study,  with  124  institutions  responding,  the  majority  of  offices 
contributing  to  faculty  development  were  the  Faculty  Development  Office  (30),  the  IRC  (28) 
and  the  Office  of  Academic  Affairs  (31).  In  most  cases,  Academic  Affairs  was  ultimately 
responsible  for  FD  programs.  In  the  1991  study,  with  data  being  provided  by  the  designated  FD 
officer,  the  data  reflected  a  similar  pattern  (see  Table  1)  with  responsibility  residing  in  the 
FD  Office  and  the  Academic  Affairs  Office. 


UNIVERSITY  ORGANIZATIONS  OFFERING  FD  WORKSHOPS 


FREQUENCY 


RESPONSIBILITY 


ORGANIZATION 


FD  Office 
IT  Academic 
Other  Academic 

Sci. 

Computer  Sci. 
Info.  Tech. 
School  of  Educ. 


24 
13 


25 
17 


11 


18 


13 


HRO 


18 


Acad.  Affairs 


42 


24 


Table  1.      Administrative  Responsibility  for  Faculty  Development 
Programs. 


Faculty  Development  Offerings  and  Attendance 

The  1990  study  provided  more  data  related  to  numbers  of  workshops/seminars  and 
attendance  than  the  1991  study.  But  the  trends  are  similar.  The  earlier  study  compared 
instructional  design-related  offerings,  media  design/use  offerings,  and  any  workshop/seminar 
related  to  computer  technology.  The  latter  study  had  five  categories-instructional  design, 
media  design/use,  computers,  interactive  technologies,  and  alternative  modes  of  instruction. 
Table  2  provides  data  on  attendance  which  is  also  graphically  displayed  in  Figure  1.  By 
eliminating  Alternative  Modes  and  combining  Computers  and  Interactive  Technologies  for  1991, 
converting  attendance  values  to  percentages  and  then  comparing  1990  with  1991  we  can  see  in 
Figure  2  that  Media  Design/Use  is  declining,  Instructional  Design  is  increasing  and  that 
Computer  Technologies  are  in  the  majority  of  FD  offerings  related  to  instructional  technology. 


ATTENDANCE  AT  FD  WORKSHOPS/SEMINARS 

INSTRUCTIONAL 

MEDIA 

COMPUTER 

HYPERMEDIA/ 

ALTERNATIVE 

DESIGN 

PROD./SEL 

USES 

INTERACTIVE 

MODES 

ATTENDANCE 

783 

337 

952 

420 

530 

•MGE  OF  TOTAL 

36 

^6 

44 

19 

25 

Table  2.     Attendance  at  FD  Workshops  and  Seminars. 
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864_ 
768 
672 
576 
480. 
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288 
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Attendance  by  Type  of  Workshop/Seminar 


IN3TRU.  DES.     MEDIA  PROD. 


£]  ATTENDANCE 
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Figure  1.     Graphic  Comparison  of  Attendance  at  FD  Workshops 
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PERCENTAGE  COMPARISON  OF  ATTENDANCE 

60, 


instructional  Design        Media  Design/Use      Computer  Technologies 


□  1 990  01991 


Figure  2.     Percentage  Comparison  of  Attendance 

Programmatic  Planning  and  Evaluation 

A  caveat  of  instructional  technology  and  design  is  that  planning  and  evaluation  are  a  part 
of  any  program  development  activity.  The  data  provided  by  faculty  development  officers  from 
65  institutions  was  disappointing  in  that  regard.  However,  faculty  input  into  program 
development  was  an  encouraging  factor.  Only  34  %  of  the  institutions  responding  indicated  that 
formal  needs  assessment  procedures  were  used  in  programmatic  planning.  Even  more 
distressing  was  that  only  26  %  had  a  formalized  program  evaluation  system.  And  43  % 
reported  that  a  formal  University  policy  existed  concerning  faculty  development.  On  the  other 
hand  all  respondents  as  a  group  indicated  that  Academic  Administration  policy  strongly  supports 
faculty  development.  The  data  regarding  programmatic  planning  is  summarized  in  Table  3. 


;          .          ■  i 

FD  PROGRAMMATIC  PLANNING 

1 

Yes 

% 

No 

% 

Is  there  a  FD  program  planning  committee? 

35 

54.00% 

30 

46.00% 

!  1 

Is  formal  needs  assessment  a  pan  of  planning? 

22 

34.00% 

43 

66.00% 

i                 i  i 

Does  the  faculty  participate  in  planning?  i 

42 

65.00% 

23 

35.00% 

1 

Is  there  a  formalized  program  evaluation  system? 

17 

26.00% 

49 

74.00% 

1                  I  i 

Is  there  a  formalized  University  policy  concerning  FD? 

20 

43.00% 

37 

57.00% 

Table  3.      Faculty  Development  Programmatic  Planning, 
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Perceptions  of  Respondents 

Each  type  of  respondent  was  asked  to  react  to  the  same  eleven  statements  related  to  the 
FD  program  at  his/her  institution.  Their  responses  were  remarkably  similar.  After 
calculating  the  Average  Weighted  Mean  for  each  group  of  respondents  on  each  item  a  Rank- 
Difference  Correlation     carman  rho)  was  computed  comparing  IT  department  heads 
perceptions  with  media  services  directors  and  with  FD  officers.  Then  media  service  directors 
were  compared  with  FD  officers.  In  each  case  there  was  an  extremely  high  positive  correlation 
among  their  cumulative  responses.  This  data  is  summarized  in  Table  4. 


RANK  DIFFERENCE  CORRELATION  AMONG  RESPONDENTS'  PERCEPTIONS 

1. 

IT  Academic  Head  Correlation  with  FD  Officer   r  =  .8636 

Extremely  positive  correlation. 

2. 

IT  Academic  Head  Correlation  with  IRC  Director  r  =  .829 

Extremely  positive  correlation. 

3. 

FD  Officer  Correlation  with  IRC  Director  r  =  .784 

Very  Positive  Correlation 

Table  4.    Summary  of  a  Rank-difference  Correlation  Among 
Respondents'  Perceptions. 

The  Average  Weighted  Value  for  the  responses  of  each  group  of  subjects  is  presented  in 
Tables  5  through  7.  Table  5  refers  to  Faculty  Development  Officers'  perceptions  about  the  FD 
program.  These  subjects  have  the  perception  that  there  is  a  shortage  of  supporting  resources, 
that  needs  assessment  does  not  play  a  significant  part  in  planning,  that  there  is  no  effective 
evaluation  plan,  that  participation  in  FD  does  not  strongly  affect  promotion  and  tenure,  and  that 
instructional  technologists  play  only  a  minor  role  in  the  Faculty  Development  Program.  There 
is  remarkable  agreement  on  these  points  among  all  three  types  of  respondents. 

FD  Officers  most  strongly  supported  the  concept  that  Academic  Administration  policy 
supports  the  FD  program.  They  also  believed  that  the  program  had  a  positive  impact  on  quality 
of  instruction  and  that  faculty  were  solicited  for  input.  IRC  Directors  and  IT  Department  Heads 
disagreed  with  FD  Officers  in  more  strongly  agreeing  with  the  concept  that  needs  assessment  is 
an  important  tool  in  planning  the  FD  program.  However,  IRC  Directors,  as  a  group,  did  not  feel 
in  agreement  with  IT  Department  Heads  and  FD  Officers  that  the  FD  program  was  having  a 
positive  impact  on  instruction. 

Conclusions.  Implications,  and  Recommendations 

Conclusions  that  may  be  made  based  on  the  findings  of  this  study  are: 
1 .  Many  departments  and  committees  on  campus  provide  workshops/seminars  that  are 
designed  to  assist  faculty  in  learning  or  improving  professional  skills. 


9 

ERIC 


7S2 
6  748 


Responses  of  Faculty  Development  Officers: 
Average  Weighted  Value 


Academic  Administration  policy 
supports  FD  program 


Faculty  actively  participates 
in  the  FD  program 


Adequate  resources  are  allocated 
to  support  the  FD  program 


FD  program  has  a  positive 
impact  on  quality  of  instruction 


FD  program  solicits  faculty 
input  on  needs 


Needs  assessment  is  an  important 
tool  in  planning  the  FD  program 


FD  program  is  proactive  in 
its  promotional  activities 


There  is  an  effective  evaluation 
plan  for  the  FD  program 


Objectives  and  goals  of  the  FD  program 
are  clearly  defined 


Participation  In  the  FD  program 
has  a  positive  impact  on 
promotion  and  awarding  tenure 

Instructional  technologists  assume 
a  primary  role  in  the  development, 
delivery,  and  administration  of  the 
FD  program 


5 
L. 


5 

U 


4  3 

-v — i- 

.\  a 
At 


2 

nr 


3\  , 


4- 


\  2 

-s-V-i- 


4 


/  2 


4- 


\ 

3  2 


4 


0 

_l 


o 

a  2 
D»  at 
*  > 

<  £ 

4.06 


3.23 


J  2.47 


j  3.71 


j  3.44 


j  2.68 


j  3.37 


0 

—I 


2.39 


3.02 


2.50 


1.89 


Table  5.    Average  Weighted  Value  of  Faculty  Development  Officers' 
Perceptions. 


2.  Most  often,  the  faculty  development  program  is  either  administered  through  the  FD 
Office  or  through  the  Academic  Affairs  office.  Ultimately,  all  such  activities  are  accountable  to 
the  Academic  Affairs  office. 

3.  The  majority  of  workshop/seminars  offered  which  relate  to  instructional  technology 
are  involved  with  computer  technology. 

4.  Workshops  related  to  media  design  and  use  are  on  the  decline. 
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Responses  of  IT  Academic  Head:  Average 
Weighted  Value 


Academic  Administration  policy 
supports  FD  program 

Faculty  actively  participates 
in  the  FD  program 


Adequate  resources  are  allocated 
to  support  the  FD  program 


FD  program  has  a  positive 
impact  on  quality  of  instruction 


FD  program  solicits  faculty 
input  on  needs 


Needs  assessment  is  an  important 
tool  in  planning  the  FD  program 


FD  program  is  proactive  in 
its  promotional  activities 

There  is  an  effective  evaluation 
plan  for  the  FD  program 


Objectives  and  goals  of  the  FD  program 
are  clearly  defined 


Participation  in  the  FD  program 
has  a  positive  impact  on 
promotion  and  awarding  tenure 

Instructional  technologists  assume 
a  primary  role  in  the  development, 
delivery,  and  administration  of  the 
FD  program 


5 

L. 


5 
U 


H  1 


+ 

3 


1 


+ 


3   \  2 


0 

-J 


4> 

cs  5 
>  ^ 


3.63 


3.47 


2.00 


3.35 


3.24 


3.25 


3.22 


2.45 


2.98 


2.58 


1.98 


Table  6.    Average  Weighted  Value  of  IT  Department  Heads'  Perceptions 

5.  Instructional  design  principles  such  as  needs  assessmant  and  evaluation  in 
programmatic  planning  of  faculty  development  programs  are  underutilized. 

6.  There  is  remarkable  agreement  among  the  three  groups  of  subjects  with  a  strong 
positive  correlation  on  the  perceptions  about  the  eleven  statements  used  to  describe  the  faculty 
development  program. 
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Responses  of  IRC  Director:  Average 
Weighted  Value 


Academic  Administration  policy 
supports  FD  program 


Faculty  actively  participates 
In  the  FD  program 


Adequate  resources  are  allocated 
to  support  the  FD  program 


FD  program  has  a  positive 
impact  on  quality  of  Instruction 


FD  program  solicits  faculty 
input  on  needs 


Needs  assessment  is  an  important 
tool  in  planning  the  FD  program 


FD  program  is  proactiva  in 
its  promotional  activities 


There  is  an  effective  evaluation 
plan  for  the  FD  program 


Objectives  and  goals  of  the  FD  program 
are  clearly  defined 


Participation  in  the  FD  program 
has  a  positive  impact  on 
promotion  and  awarding  tenure 

Instructional  technologists  assume 
a  primary  role  in  the  development, 
delivery,  and  administration  of  the 
FD  program 


4> 

M 
« 

<5 


4.83 


2.76 


2.37 


2.70 


2.91 


J. 46 


3.20 


2.28 


2.70 


2.39 


1.81 


Table  7.    Average  Weighted  Value  of  IRC  Directors'  Perceptions. 

It  may  be  implied  that  instructional  technologists  and  instructional  design  principles 
are  being  under  utilized  in  the  development,  administration,  and  delivery  of  faculty 
development  program  offerings.  The  implications  of  this  indicate  a  need  for  a  more  proactive 
stance  from  instructional  technologists  in  participation  in  the  FD  program.  In  addition,  these 
professionals  need  to  develop  greater  visibility  on  campus  in  their  modeling  of  good  teaching  and 
helping  others  increase  their  productivity  through  technology. 

It  is  recommended  that  a  more  comprehensive  study  be  made  related  to  faculty 
development  with  on-site  visits  cooperatively  working  with  appropriate  campus  personnel  and 
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carefully  monitoring  the  FD  program  regarding  planning  activities,  needs  assessment, 
evaluation  and  kinds  and  numbers  of  workshops  with  attendance  figures.  Standards  should  be 
developed  by  which  Faculty  Development  programs  can  be  evaluated  and  model  programs 
recognized.  What  makes  an  effective  program?  What  can  be  done  to  ensure  that  the  program 
remains  effective? 
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Introduction 

Instructional  systems  design  (ISD)  or  the  systems  approach  is  being  used  in  the 
training  in  the  U.S.  to  an  increasing  degree,  and  it  is  generally  successful.  However, 
it  is  not  well-known  as  to  whether  this  approach  is  applied  in  the  other  countries. 
One  major  issue  in  ISD  is  its  applicability  to  different  contexts,  particularly  to 
different  countries.  Each  country  has  its  own  approaches  and  ideas  regarding  training. 
There  may  be  situations  where  ISD  is  effective,  and  ones  where  it  does  not  quite  fit, 
at  least  not  without  modification  of  the  ways  in  which  it  is  generally  represented  in  the 
literature  of  the  U.S.A.  and  Europe,  Before  discussing  the  effectiveness  of  ISD  and 
implementing  it,  an  essential  step  is  to  understand  the  context  where  the  training  is 
taking  place. 

In  Japan,  the  terms  "ISD"  and  "the  systems  approach"  are  not  found  in  the  field  of 
training.  However,  tills  does  not  mean  that  the  concepts  and  processes  or  ISD  are  not 
being  used.  It  is  possible  that  certain  elements  of  ISD  are  being  implemented  without 
being  identified  as  such.  It.  will  be  useful  to  have  a  detailed  description  of  what 
approach  is  being  used  in  training  as  well  as  an  overall  picture  of  the  training  in 
Japan.  This  research  will  help  us  to  understand  the  current  use  of  ISD,  or  elements  of 
it,  and  the  possibility  of  its  future  use. 

Research  Questions 

The  purpose  of  this  research  is  to  obtain  a  picture  of  approaches  to  training  in 
Japan,  focusing  on  its  use  of  ISD.  This  will  serve  to  assist  in  answering  two 
questions:  "What  kind  of  training  is  conducted  in  Japanese  companies?"  and  "To  what 
degree  is  ISD  implemented?"  This  will  provide  both  practitioners  and  researchers  with 
valuable  information  regarding  current  practices  and  future  potential  for  ISD. 

Method  of  the  Research 

The  survey  method  was  selected  as  the  most  appropriate  for  the  purpose  of  the 
research,  because  it  enables  the  researchers  to  cover  a  larger  sample  than  any  other 
method.  As  for  sampling  strategy,  stratified  random  sampling  was  employed  to  ensure 
obtaining  a  sufficient  sample  within  different  industries.  Six  major  industries  were 
chosen  because  their  counterparts  in  the  U.S.  tend  to  have  strong  internal  training 
departments:  banks,  auto  manufacturing,  electric  machinery,  wholesale  stores, 


'This  paper,  which  was  used  as  a  handout,  is  a  synopsis  of  the  full-length  paper. 
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insurance  and  securities,  and  transportation.  The  researchers  selected  121  companies 
from  about  300  companies  in  the  six  industries  which  were  listed  in  the  First  Section 
of  the  stock  exchange  in  Japan  in  1990.  Before  administering  to  121  companies,  the 
questionnaire  was  pilot  tested  in  two  Japanese  companies  and  revised.  The  study  was 
conducted  in  July  and  August,  1991. 

Results  of  the  Research 

Forty-five  companies  out  of  121  responded  (6  out  of  20  in  banks,  7  out  of  20  in 
auto  manufacturing,  11  out  of  21  in  electric  machinery,  8  out  of  20  in  wholesale 
stores,  6  out  of  20  in  insurance  and  securities,  and  7  out  of  20  in  transportation). 
Hence,  the  response  rate  was  37.5% 

The  first  question  was,  "Considering  all  of  your  employees,  when  does  your 
company  use  education  and  training  done  off  the  job  (off-JT)  more  frequently  than 
education  and  training  done  on  the  job  (OJT)?"  Off-JT  refers  to  training  that  is 
offered  internally  by  the  company  at  a  training  location  and  requires  the  employee  to 
leave  the  work  site  while  attending  training.  Table  1  indicates  the  relative  use  of 
off-JT  and  OJT.  Through  the  industries,  off-JT  is  more  often  used  than  OJT  for 
orientation  of  new  employees.  Half  of  the  companies  (22  out  of  44)  use  off-JT  more 
often  than  OJT  as  a  means  of  employees'  self-development.  As  for  training  directly 
related  to  the  present  job,  the  tendency  to  use  OJT  is  strong. 


Table  1— Companies  using  off-JT  more  frequently  than  OJT  for  each  training  objective 


Training  objective 

#  of  companies 
(n=44) 

Percent 

Orientation  of  new  employees 

28 

64% 

Providing  employees  with  an  opportunity 
for  self-development 

22 

50% 

Helping  employees  qualify  for  future  jobs 
within  the  organization 

19 

43% 

Helping  employees  perform 
their  present  job  well 

16 

36% 

Informing  employees  of  technical  and 
procedural  changes 

8 

18% 
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The  next  question  was,  "Which  method  or  material  does  your  company  use  most 
frequently  each  for  technical  skills  and  interpersonal  skills?  Place  a  1  after  the  most 
frequently  used  approach,  then  indicate  the  2nd  and  3rd  most  frequent  approach."  As 
a  whole,  the  most  frequently  used  training  methods  for  technical  skills  are  practical 
training,  lecture,  and  group  discussion  in  that  order  (Table  2).  This  is  true  across  the 
industries,  except  for  wholesales  where  group  discussion  and  role  play  /  case  study 
are  the  two  more  frequently  used.  A  different  tendency  is  found  in  training  methods 
for  interpersonal  skills  where  role  play  /  case  study,  group  discussion,  and  practical 
training  are  the  most  frequently  employed.  This  is  true  in  each  industry,  except  for 
transportation  where  lecture  occupies  the  third  place. 


Table  2  ---  Frequency  in  the  use  of  training  methods 
(3  points  for  most  frequent,  2  for  the  next,  1  for  the  next; 


Training  Method 

Technical 

Inter- 
personal 

Total 

Group  discussion 

32 

(3) 

68 

(2) 

100 

Practical  training 

55 

(1) 

42 

(3) 

97 

Lecture 

53 

(2) 

38 

(4) 

91 

Role  play  /  case  study 

17 

(5) 

70 

(1) 

87 

A-V  media 

23 

(4) 

7 

(5) 

30 

Simulations 

6 

(6) 

6 

(6) 

12 

Correspondance 

2 

(8) 

1 

(8) 

5 

Games 

2 

(9) 

3 

(7) 

5 

Computers 

4 

(7) 

0 

(9) 

4 

Table  3  summarizes  the  responses  to  the  question,  "How  does  your  company  obtain 
training  programs?"  Developing  programs  within  the  company  is  the  most  frequent 
Lnd  obtaining  training  from  outside  sources  is  the  next  frequent  way.  This  tendency  is 
found  all  through  the  industries.  Forty-one  companies  out  of  44  indicates  developing 
in  own  company  as  the  most  frequent  way. 


The  following  questions  were  asked  to  companies  developing  at  least  some  of  their 
own  programs.  The  first  question  was,  "Who  develops  and  delivers  your  training 
programs?"  Table  4  illustrates  that  training  department  plays  a  central  role  in  both 
developing  and  delivering  training.  Most  of  the  companies  indicated  that  the  training 
department  works  with  either  an  SME  or  supervisors.  All  the  companies  in  banks  and 
transportation  stated  that  the  same  sections  are  in  charge  of  development  and  delivery 
of  training.  However,  some  of  the  companies  in  other  industries  include  different 
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sections  for  development 
and  delivery  (such  as 
SMEs  and  training 
department  for 
development  and  SMEs 
and  supervisors  for 
delivery).  The  use  of 
SMEs  external  to  the 
company  and  that  of 
SMEs  internal  to  the 
company  are  about  the 
same  in  the  frequency. 


Three  examples  of  the 
ISD  model  were 

provided  with  a  question,  "Does  your  company  have  a  similar  model  or  use  a  similar 
procedure  without  a  written  model?"  Only  two  companies  indicated  that  they  have  a 
model  similar  to  the  ISD  model  (Table  5).  About  70%  of  the  companies  identified  that 
they  use  a  similar  procedure,  but  do  not  have  a  specific  model  to  follow.  This 
tendency  is  found  across  the  industries.  Among  those  who  are  using  a  different 
procedure,  half  of  them  recognized  such  a  model  would  be  useful  in  finding  out  real 
needs  and  responding  to  them  in  a  timely  manner  or  useful  as  a  check  list.  Those  who 
find  that  the  model  does  not  look  helpful  pointed  out  a  lack  of  flexibility. 


Table  4  —  Who  are  in  charge  of  developing  and  delivering  training  programs 


Develop 

Deliver 

Who  in  charge 

#  of  companies 
n=45 

Percent 

#  of  companies 
n=45 

Percent 

SME  (external) 

24 

53% 

27 

60% 

SME  (internal) 

25 

55% 

31 

69% 

Supervisors 

10 

22% 

15 

33% 

Training  department 

40 

89% 

35 

78% 

Others 

0 

0% 

1 

0% 

While  some  companies  use  the  ISD  procedure  for  all  of  their  training  programs, 
others  use  the  procedure  selectively  CTable  6).  Differences  were  found  among  the  ' 
industries  as  to  which  training  categories  employ  the  procedure  and  which  do  not.  For 
example,  in  entry  level  training,  less  than  half  of  electric  companies  and  the  half  of 


Tabic  3  —  Where  to  develop  training  programs 

'3  points  for  the  most  frequent,  2  for  the  next,  1  for  the  next) 


Points 

Where  to  develop  (#  of  companies  «  44) 

In  own  company  127 

In  affiliate  company  28 

From  outside  80 
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transportation  companies  use  the  ISD  procedure,  while  all  the  banks  and  wholesales 
companies  use  it.  For  research  and  engineering,  all  the  companies  who  conduct  such 
a  training  use  the  procedure.  The  use  of  the  procedure  is  the  lowest  in  training  for 
entry  level  workers  across  the  industries,  although  such  a  training  is  identified  as  the 
most  frequent.use  of  off-JT. 

Short  answer  questions 
were  asked  to  obtain  a 
closer  look  at  the  training 
situation.  The  first  question 
was,  "How  do  you  decide 
needs  for  developing  a  new 
program  or  revising  a 
present  program?" 
Techniques  universally 
identified  were 
questionnaires  for  previous 
trainees,  interviews  with 
people  in  relevant  levels  and 
sections,  and  meeting  with 
supervisors  or  other 
important  people.  Among 
others  are  breaking  down 
needs  from  a  business  plan 
and  collecting  information 
from  outside.   As  for  the 
next  question.  "How  do  you 
determine  what  the  content 
of  a  course  will  be?,"  almost  no  answer  2  20% 

all  the  responses  indicated  -  

that,  after  finding  out  needs, 
the  training  department 

holds  a  meeting  either  by  itself  or  joined  by  a  section  of  target  audience.  As  for  the 
question,  "Is  there  a  formal  process  or  tool  for  the  design  and  development  of  a 
training  material?,"  no  companies  stated  that  they  have  such  a  procedure  or  tool  for 
training  materials.  It  seems  that  they  develop  or  revise  materials  case  by  car  ;, 
following  the  previous  example.  A  few  companies  indicated  that  they  consult  with 
outside  information  source  such  as  journals  and  consultants. 

The  next  question  was  "How  and  who  conducts  training  evaluation?"  The  most 
popular  way  of  evaluation  is  a  combination  of  questionnaire  for  trainees  on  the 
training  program  and  interview  with  their  supervisors  on  their  job  performance  after 
the  training.  Some  companies  ask  supervisors  to  write  a  report  on  subordinates  who 
went  through  a  training  program, 

Various  effoi-s  were  reported  on  "What  kind  of  approach  do  you  use  for  improving 
the  effectiveness  of  training  programs?"  Follow-ups  (evaluation  and/or  training),  a 
variety  of  training  methods  as  well  as  curriculum,  and  an  extensive  needs  assessment 
are  the  three  most  identified  approaches. 


Table  5  —  Companies  using  an  ISD  model  or  a  similar  procedure 


Use  of  ISD  procedure 

#  c  f  companies 
(n=42) 

Percent 

Own  a  model 

2 

5% 

Use  a  similar  procedure 

29 

69% 

Use  a  different  procedure 

10 

24% 

For  companies  using  a  different  procedure, 
does  this  kind  of  model  look  helpful? 

Helpful 

4 

40% 

Not  helpful 

4 

40% 
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Discussion 


Much  of  corporate 


Table  6  ~  Use  of  ISD  procedure  for  each  training  categories 


training  and  education  is  ^    k  /u  •    y*t>  a 

a        *  ATm    „  T  *  Percent  (#  using  ISD  procedure 

?L        1         '  *  ^ducting  a  particular  training) 


and  self-development,  each 
of  which  has  its  own  role. 
Japanese  companies  put 
more  emphasis  on  OJT 
traditionally;  however,  this 
research  shows  that  OJT  has 
established  a  certain  position 
in  large  companies.  It  is 
used  primarily  for  entry 
level  training  and  for 
opportunities  for  self 
development. 


Entry  level 

67% 

(20/30) 

Production 

94% 

(15/16) 

Clerical  work 

74% 

(14/19) 

Sales  work 

87% 

(20/23) 

Management  skills 

90% 

(29/31) 

Research  and  engeneering 

100% 

(12/12) 

At  this  stage,  it  is  not 
clear  how  effective  formally  developed  training  programs  are  compared  to  tnditional 
OJT.  Since  most  of  the  companies  in  this  study  stated  that  developing  train'ng 
programs  on  their  own  with  the  training  department  as  its  center  is  a  primary 
approach  for  development,  how  to  develop  programs  is  crucial. 

The  majority  of  the  companies  studied  identified  that  they  an  using  a  procedure 
similar  to  the  ISD  model.  However,  a  closer  look  reveals  that  what  they  are  doing  can 
be  called  a  general  systems  approach  to  problem  solving,  rather  than  ISD.  The 
companies  follow  general  steps  of  systems  approach  to  problem  solving,  but  not 
detailed  steps  of  ISD,  when  developing  training  programs. 

The  fact  that  systems  approach  has  already  been  implemented  and  also  general 
positive  reaction  toward  ISD  models  suggest  that  ISD  can  be  introduced  successfully, 
as  a  specific  application  of  the  systems  approach  to  instruction. 

Here,  special  attention  should  be  paid  to  the  possible  perception  of  ISD  as 
inflexible,  which  possibly  comes  from  the  way  ISD  is  generally  described  (flowchart, 
step-by-step  task  list,  and  the  like).  It  is  recommended  that  ISD  be  introduced  as  a 
general  concept  of  developing  instruction,  not  as  a  specific  procedure  to  be  followed. 

Finally,  in  closing,  it  is  important  to  note  some  differences  between  the  United 
States  and  Japan  in  terms  of  organizational  structures  that  influence  the  adoption  and 
utilization  of  ISD  in  Japan.  In  Japan,  corporations  tend  to  exercise  a  high  degree  of 
proprietary  control  over  training  along  with  their  other  internal  management  and 
manufacturing  processes.  Consequently,  they  seldom  bring  in  outside  consultants  or 
vendors  to  develop  training  although  they  may  bring  in  outsiders  to  present  off-the- 
shelf  training.  Furthermore,  there  tend  not  to  be  working  relationships  between 
professors  in  the  universities  and  business  and  industry. 

These  factors  limit  the  likelihood  of  full  scale  adoptions  of  ISD  processes  and 
supporting  technologies  in  the  near  future,  but  there  do  appear  to  be  changes 
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occurring.  Professors  in  some  of  the  leading  educational  technology  programs  are 
conducting  innovative  research  on  technology  applications  in  instruction,  and  are 
devoting  more  attention  to  teaching  ISD  in  their  graduate  programs.  Furthermore, 
some  of  them  are  knowledgeable  of  the  utilization  of  ISD  and  technology  in  education 
in  corporations  in  the  United  States.  These  trends,  together  with  other  changes  that 
are  occurring  in  the  younger  generations  of  Japanese  with  regard  to  their  attitudes 
toward  careers,  indicate  that  Japanese  will  be  looking  more  carefully  at  training 
development  options  that  are  more  efficient  than  the  traditional  OJT  approaches. 
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Choosing  a  Display  Format  for  InstructionalMultimedia 

Two  Screens  vs.  One 


The  technology  for  capturing,  storing  and  displaying  multiple  media  with  the 
computer  has  made  revolutionary  strides  within  the  past  few  years.  Given  the 
wealth  of  choices  now  available  to  the  designer  of  multimedia  courseware — which 
must  be  balanced  against  the  technological  constraints  of  a  given  computer  system — 
the  decision  about  presenting  media  in  an  instructional  situation  is  more 
problematic  than  ever.  Of  the  many  available  alternatives  for  displaying 
information  to  the  learner,  what  does  the  designer  need  to  know  in  order  to  make  a 
decision  that  will  further  the  project  goals?  This  presentation  will  first  give  a  brief 
review  of  multimedia  display  technology.  The  second  section  will  describe  the 
technical,  theoretical  and  practical  factors  that  should  be  considered  in  making  a 
decision  about  a  display  system.  Technical  decisions  involve  consideration  of  the 
trade-offs  among  the  resources  of  a  particular  computing  system.  Theoretical  factors 
draw  on  research  and  guidelines  from  a  wide  variety  of  fields  that  include  learning 
theory  and  instructional  design,  human-computer  interface  design  (human  factors 
research),  visual  learning  and  media  selection.  The  third  section  will  present  a  brief 
description  of  the  decision-making  process  that  went  into  the  design  of  a 
multimedia  workstation  for  a  plant  biology  laboratory. 


Factors  in  Decision-making 

How  do  we  make  decisions  about  when  to  use  the  various  media  within  a  single 
courseware  program?  A  traditional  area  of  instructional  design  and  development 
concerns  media  selection,  and  a  number  of  useful  models  have  been  developed  to 
aid  in  choosing  the  appropriate  media  to  deliver  an  instructional  message  (Reiser  & 
Gagne,  1982).  The  issues  raised  and  discussed  in  the  rationale  for  these  models  are 
highly  relevant  in  the  choices  of  the  individual  media  used  within  a  multimedia 
system.  However,  almost  all  of  the  relevant  research — and  the  resulting  established 
and  tested  models — concerns  single  media  used  in  isolation,  and  the  models  were 
developed  well  before  the  advent  of  multimedia.  The  question  for  multimedia,  on 
the  other  hand,  is  of  a  different  nature:  in  the  face  of  the  incredibly  rich  variety  of 
choices,  how  does  a  developer  or  designer  choose  and  mix  different  media  to 
achieve  the  best  effect?  This  is  a  different  sort  of  question,  and  it  is  the  central  issue 
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in  this  article.  There  are  three  perspectives  on  this  problem:  one  based  on 
theoretical  principles  of  learning  and  instructional  design,  the  second  dealing  with 
technical  trade-offs  within  a  computer  system,  and  the  third  purely  practical. 


Theoretical  Factors 

Due  to  its  recent  emergence,  there  are  no  well-demarcated  boundaries  to 
multimedia  as  a  field  of  research.  Some  authors  (cf.  DeBloois,  1989)  have  wisely 
recommended  that  we  should  refrain  from  spelling  out  strict  guidelines  for  usage, 
because  it  may  hinder  experimentation  in  the  field.  However,  several  areas  of  past 
research  are  relevant  in  making  decisions  about  the  display  of  multimedia, 
although  it  may  require  effort  and  imagination  in  order  to  transfer  results  to  this 
new  field.  These  areas  include  learning  theory  and  instructional  design,  media 
selection,  visual  learning,  and  human-computer  interface  design  (from  human 
factors). 

Instructional  design  and  learning  theory  are  closely  interwoven  (cf.  Gagne,  1985; 
Gagne  &  Briggs,  1979)),  and  have  had  a  seminal  influence  on  the  development  of 
computer-assisted  instruction  and  interactive  video  (Jonassen,  1988;  Hannafin  & 
Rieber,  1989;  Hannafin  &  Phillips,  1987),  from  which  the  current  conception  of 
multimedia  is  derived  and  closely  allied.  Although  multimedia  and  CAI  can  make 
a  useful  contribution  during  each  part  of  the  instructional  process,  the  method  of 
display  is  more  consequential  in  some  than  others.  The  framework  provided  by 
ROPES+  (Hannafin  &  Rieber,  1989;  Hooper  &  Hannafin,  1988),  a  technology- 
oriented  "meta-model"  for  instructional  design  based  on  applied  cognitivism  and 
information  processing  theory,  is  especially  useful  for  application  of  design 
principles  in  the  decisions  involving  display  technology.  The  following  discussion 
of  attention,  encoding  and  retrieval  and  presentation  will  identify  key  areas  of  the 
instructional-learning  process  in  which  the  mode  of  display  has  the  most  salient 
areas  of  influence. 

Attention  as  a  Learner  Resource 

Attention  is  a  key  construct  in  learning  and  instruction  (Klatzky,  1980;  Gagne,  1985) 
and,  for  that  matter,  in  any  form  of  communication.  Workers  in  the 
communications  industry,  especially  in  the  advertising  sector  of  broadcast  media, 
wage  a  constant  battle  to  capture  the  public's  attention  and  direct  it  favorably  toward 
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their  product  and  message.  Cognitive  learning  theorists  have  demonstrated  that 
attention  is  a  resource  with  limited  capacity  that  can  be  allocated  to  only  a  few 
processes  at  one  time  (Kahneman,  1973;  Norman,  1976;  Klatzky,  1980).  In  addition, 
more  attention  is  required  for  unfamiliar  material,  such  as  normally  occurs  in  an 
instructional  situation.  As  such,  the  designers  of  multimedia  must  be  constantly 
aware  that  they  are  also  working  to  gain,  hold,  control  and  direct — and  even 
provide  relief  for— the  learner's  attention.  This  power  should  not  be  taken  lightly, 
and,  indeed,  it  implies  a  serious  responsibility  for  the  cognitive  status  of  the  learner. 

In  the  events  of  instruction  model  (Gagne,  1985),  gaining  attention  serves  as  the 
"wake-up"  call  for  learning  it  prepares  the  stage  for  learning  and  can  be  coordinated 
with  orienting  devices,  such  as  advance  organizers  (Ausubel,  1963)  and  stimulating 
the  recall  of  previously-learned  information.  Multimedia,  because  of  its  novelty,  is 
currently  a  premier  device  for  attention-getting.  However,  attention-getting  should 
be  used  cautiously  (getting  does  not  guarantee  holding  attention).  For  example,  it 
would  not  be  prudent  to  waste  a  large  part  of  the  system  resources  on  a  flashy 
opening  sequence,  and  then  use  a  text-only  presentation  during  the  remainder  of 
the  instructional  program.  It  is  easy  to  irritate  users  and  lose  their  attention  if  high 
expectations  are  not  fulfilled;  the  designer  should  be  careful  to  deliver  on  all 
promises,  or  the  result  could  easily  be  a  net  negative  effort, 

Creative  uses  of  multimedia  can  facilitate  the  gaining,  directing  and  holding  of 
learner  attention.  For  example,  orienting  activities  (such  as  advance  organizers  or 
presentation  of  objectives  prior  to  instruction)  are  "mediators  through  which  new 
information  is  presented"  (Ilannafin  &  Phillips,  1987)  and  prepare  the  learner  for 
forthcoming  instruction.  Advance  organizers  (Ausubel,  1963)  have  mixed  claims  of 
success,  it  is  often  asserted,  because  they  are  not  applied  according  to  prescription 
(Ausubel,  1977;  Mayer,  1979),  which  is  at  a  "higher  level  of  abstraction,  generality 
and  inclusiveness"  than  the  subsequent  content  (Ausubel,  1963).  It  has  been 
recognized  that  video  is  not  an  ideal  medium  for  presenting  detailed  material,  but  is 
better  used  for  broader,  abstract  material,  possibly  with  an  emotional  appeal. 
Therefore,  a  short,  abstract  video  segment  may  serve  well  as  the  medium  for  an 
advance  organizer,  and,  similarly,  for  a  lesson  summarizer.  On  the  other  hand, 
AIME  research  (Salomon,  1984;  Cennamo,  Savenye,  &  Smith,  1991)  has  shown  that 
television,  because  it  is  usually  perceived  as  easy  to  understand  and  places  few 
demands  on  viewers,  can  cue  learners  to  invest  less  mental  effort  in  learning 
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program  content.  Consequently,  multimedia  designers  would  be  well  advised  to 
avoid  the  look  and  feel  of  "television"  in  the  design  of  their  programs;  for  one 
thing,  this  would  seem  to  imply  that  they  should  avoid  long,  linear  video  sequences 
and  should  increase  interactivity  whenever  possible. 

The  use  of  multiple  media  can  be  useful  in  providing  the  variety  within  a 
presentation  that  is  important  for  maintaining  and  holding  attention.  Use  of  the 
different  media  can  strengthen  the  power  of  CAI  to  vary  viewpoints,  the  pace  of  a 
presentation  and  provide  numerous  opportunities  for  interaction  and  practice.  It 
has  also  long  been  recognized  that  each  medium  has  its  own  attribu  tes  and  symbol 
system  that  directly  or  indirectly  impacts  learners,  which  renders  it  more  or  less 
suitable  for  different  instructional  outcomes  and  content  (Salomon,  1979;  Salomon 
&  Gardner,  1986).  In  addition,  different  media  assume  and  develop  different  skills 
(McCluhan,  1965)  at  the  same  time  that  their  messages  converge  as  to  the  knowledge 
that  they  convey  (Bruner  &  Olson,  1973).  For  this  reason,  media  selection  has 
evolved  into  a  well-established  decision-making  process,  and  a  number  of  selection 
models  exist  that  have  a  variety  of  formats  and  take  different  learner,  task,  and 
instructional  variables  into  account.  However,  almost  without  exception,  these 
models  were  developed  with  the  assumption  that  individual  media  were  to  be  used 
in  isolation.  The  advent  of  multimedia  changes  the  basic  ground  rules  of  media 
selection  and  usage,  and  the  changes  currently  underway  point  toward  re-opening 
this  important  field  of  research  (Locatis,  Charuhas  &  Banvard,  1990).  Kozma  (1991) 
notes  that  choices  about  media  selection  were  formerly  macrolevel  decisions,  but 
with  multimedia  such  decisions  can  be  made  more  frequently  and  selectively  at  the 
microlevel,  which  opens  up  the  possibility  of  selecting  media  based  on  individual 
learner  needs  and  preferences. 

A  basic  premise  of  this  article  is  that  the  multimedia  devices  that  are  the  most 
powerful  in  terms  of  gaining  attention  also  tend  to  be  the  most  expensive  in  terms 
of  system  resources.  Digital  motion  video  and  24-bit  color  animation,  it  has  already 
been  noted,  require  extremely  large  file  sizes  and  extensive  CPU  loading.  These 
formats  cannot  be  used  too  freely  on  most  systems  available  today.  The  list  of 
presentation  formats  below  represent  an  assertion  by  the  author — based  on 
intuition,  experience  and  available  research — of  a  range  of  presentation  devices 
scaled  from  least  attention-holding  (1)  to  most  attention-holding  (6).  It  is  a  relative 
scale,  in  that  if  a  screen  contained  two  items  with  different  formats  that  competed 
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for  attention,  the  item  that  was  higher  on  the  scale  would  tend  to  hold  the 
viewer/user's  attention  more  effectively.  This  list  represents  testable  hypotheses; 
results  could  be  used  in  a  quantitative  trade-off  analysis  (Norman,  1987). 

Attention-getting/holding  devices  from  least  to  most  attention-holding 

1.  text  only 

2.  Static  visual /graphic  (varying  by  color  and  size) 

3.  Animated  visual/graphic  (silent) 

4.  Sound  alone 

5.  Sound  +  Movement 


An  additional  assertion  is  the  following  list,  which  represents  items  that  tend  to 
create  an  unpleasant  discord  in  the  user's  multimedia  experience,  and  therefore 
have  a  net  negative  effect  in  attention  and,  therefore,  learning.  These  are  presented 
in  no  particular  order. 

0.  Sound  vs.  movement 

0.  Incomprehensible,  badly-designed  static  or  moving  visual 

0.  Obnoxious  sounds 

0.  Repetitive  image,  s^und,  etc. 

0.  Very  slow  movement  (or  response  to  input)  that  forces  the  user  to  wait  before 
progresjing 

Research  in  the  relative  importance  of  television  production  factors  is  useful  for  the 
video  and  audio  aspects  of  multimedia  (Wright  &  Huston,  1983).  For  example,  it 
has  been  found  that  most  learning  occurs  when  audio  and  video  are  redundant,  that 
is,  are  synchronized  and  repeat  and  reinforce  the  same  content;  when  audio  and 
video  interfere  with  each  other,  learners  consistently  shift  their  attention  to  the 
video  at  the  expense  of  the  audio  trade  (Hanson,  1989). 

Encoding  and  retrieval  are  crucial  events  in  instruction;  they  might  even  be 
described  as  the  defining  operations  of  the  learning  process.  Information  must  be 
encoded  in  long-term  memory  in  such  a  way  that  it  can  be  reficved  when  needed; 
otherwise,  the  information  is  inaccessible  and  therefore  useless,  a  situation  that  has 
been  described  as  inert  knowledge  (Whitehead,  1929).  For  example,  Paivio's  (1979) 
dual-coding  hypothesis  suggests  that  encoding  can  take  place  in  two  channels,  one 
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visual  and  the  other  verbal.  Research  in  this  area  shows  that  the  more  concretely 
information  is  presented,  the  easier  it  is  to  picture  and  remember;  the  more  abstract, 
the  harder  it  is  to  picture  and  remember.  One  of  the  strengths  of  multimedia, 
obviously,  is  its  wealth  of  techniques  for  presenting  information  in  concrete  form. 

A  key  principle  asserted  by  the  ROPES+  model  is  that  distinctiveness  at  encoding 
(which  should  to  be  distinguished  from  image  clarity)  will  h*  related  to  importance, 
which  will  facilitate  the  retrieval  of  knowledge.  An  important  conclusion  to  be 
drawn  from  th's  principle  is  that  designers  should  save  their  most  distinctive 
presentation  power  for  key  concepts  and  ideas.  '  A  related  principle  is  spread  of 
activation,  in  which  superordinate  concepts  set  the  stage  for  learning  related, 
subordinate  concepts.  This  is  the  sam     ittern  as  is  used  with  orienting  activities: 
after  attention  is  directed  to  the  key  concept,  subsequent  subordinate  concepts  will  be 
associated  with  the  superordinate.  A  conclusion  to  be  drawn  here  is  that  expensive 
system  resources  (e.g.,  high-attention  devices,  such  as  animation  drawn  in  24-bit 
color,  or  full  motion  video)  should  be  reserved  for  cuing  learners  as  to  the 
importance  of  superordinate  concepts;  simpler  displays  will  then  suffice  in  the 
presentation  of  related  but  subordinate  concepts.  In  addition,  the  association  of 
knowledge  with  the  context  in  which  it  is  to  be  used  is  important  in  the  encoding 
process,  which  then  can  lead  to  its  appropriate  retrieval;  although  text  can  describe 
context,  the  realism  with  which  video  can  demonstrate  context  is  one  of  its  most 
powerful  features  (Hamilton  &  Taylor,  in  press). 

The  presentation  of  information  has  been  given  the  greatest  amount  of  attention  in 
media  and  CAI  design,  although,  as  we  have  seen  here,  it  is  but  one  stage  in  the 
instructional  process.  A  great  deal  of  research  has  been  done  in  this  area,  such  as  the 
extensive  work  in  visual  learning  (Dwyer,  1972),  text  and  screen  design  (e.g., 
Morrison,  Ross,  Schultz,  &  O'Dell,  1989),  and  human  factors  interface  design 
(Rubinstein  &  Hti&h,  1984;  Hancock,  1987;  Laurel  &  Mountford,  1990),  which  has 
resulted  in  useful  guidelines.  Due  to  the  recent,  rapid  advancements  in 
multimedia,  however,  most  of  this  work  provides  little  specific  help  to  the 
multimedia  designer/developer,  although  some  principles  do  transfer  to 
presentations  of  mixed  media.  For  example,  research  in  visual  learning  shows  that 
the  addition  of  visuals  often  helps  comprehension  of  many  different  types  of 
information.  However,  an  increase  in  visual  complexity  (e.g.,  greater  realism — 
photographs)  in  visual  illustrations  rareiy  helps  in  comprehension,  especially  in 
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higher-order  learning  (Dwyer,  1972).  This  research  can  help  in  decisions  about 
presenting  high-end  graphics  using  244*  color,  which  are  costly  in  terms  of  system 
resources.  These  effects  can  vary  according  to  learner  characteristics  (e.g.,  Canelos, 
Taylor  &  Gates,  1980).  Of  course,  when  deciphering  visual  complexity  is  an 
instructional  objective-such  as  when  biology  students  must  identify  details  on  a 
photomicrograph  of  plant  tissue-the  capability  to  display  high  image  quality 
becomes  a  high  priority. 

Summary:  Obviously,  a  great  deal  of  existing  research  can  inform  the  decisions 
regarding  display  devices  in  multimedia.  As  yet,  however,  there  is  very  little 
research  that  applies  specifically  to  key  attributes  of  multimedia,  such  as  the  effects 
of  mixing  media,  efforts  at  directing  attention  within  the  screen  or  screens,  use  of 
multiple  versus  combined  screens,  or  other  factors  discussed  here.  This  is  primarily 
due  to  the  fact  that  the  field  itself  has  only  very  recently  come  into  existence;  not 
only  have  researchers  had  little  time  to  do  research,  but  the  rapid  fluctuations  of  the 
field  have  made  it  difficult  to  set  research  priorities.  Until  relevant,  specific  research 
is  performed  and  results  in  viable,  useful  guidelines,  practitioners  must  rely  on 
related  research  findings,  experimentation  and  intuition. 

Technological  Factors 

A  Brief  Technology  Review 

The  increase  in  capability  of  the  personal  computer  has  been  a  story  of  sure  and 
steady-albeit  extremely  rapid-progress  in  processing  power,  clock  speed, 
bandwidth,  etc.,  and  computer  users  have  watched  on-screen  displays  change  from 
all  text  with  very  simple  monochrome  graphics  to  photorealistic  images  and 
sophisticated  audio-enhanced  animation.  The  fundamental,  ground-breaking 
changes,  however,  are  most  evident  when  we  review  the  way  that  video  has  been 
associated  with  the  computer  display. 

Managed  Video  vs.  Integrated  Video 

In  the  early  days  of  microcomputer-based  interactive  videodisc  (circa  1980),  an 
interactive  video  presentation  was  limited  to  two  screens:  a  computer  monitor  lor 
digital  information  (text  and  computer  graphics),  and  a  video  monitor  for  a 
television  signal,  which  was  played  back  from  a  videodisc  or  videotape 
(Romiszowski,  1986,  p.  384).  Insofar  as  the  video  was  concerned,  the  computer 
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functioned  as  a  primarily  control  device  for  finding  and  displaying  either  single 
video  frames  (as  still  images)  or  sequences  of  frames  (as  motion  video)  on  the  video 
monitor.  From  the  beginning,  there  has  always  been  an  strong  movement  toward 
combining  the  two  displays,  but  the  fundamental  differences  between  the  computer 
and  television  signals— different  scan  rates,  retrace  and  blanking  speeda, 
bandwidths,  overscan/underscan,  etc.— made  the  marriage  difficult.  These 
technical  differences  are  many  and  complex,  and  it  is  impossible  to  cover  them  here. 
Two  solutions  to  this  fundamental  incompatibility  of  the  two  signals — either 
overlay  boards  or  special  monitors— allowed  the  computer  signal  to  be 
superimposed  over  (but  not  actually  mixed  with)  the  television  image.  The 
epitome  of  this  solution  was  the  original  IBM  InfoWindow  display  system,  which 
was  actually  a  television  and  computer  monitor  combined  within  a  single  box  and 
screen.  As  might  be  expected,  such  elaborate  solutions  added  considerable  cost  to  a 
microcomputer  system,  and  the  most  affordable  applications  of  interactive  video 
continued  to  use  the  two  screen  system. 

In  the  last  few  years,  a  number  of  engineering  advancements,  in  the  form  of 
sophisticated  yet  affordable  add-on  boards,  have  advanced  the  merger  of  video  with 
the  computer  system  (Wells,  1989).  These  "desktop  video"  boards  (ColorSpace 
Ili/FX,  Targa  boards,  NuVista,  etc.)  offered  several  useful  features,  which  could  be 
grouped  into  three  functional  areas.  They  could  digitize  a  single  video  frame  (at 
various  resolutions  and  color  depths),  which  could  then  be  stored  and  used  in  the 
same  manner  as  a  standard  graphics  file.  Second,  they  could  convert  the  computer 
signal— such  as  computer  graphics  and  animation— into  a  television  signal  (NTSC) 
for  output  to  videotape.  Finally,  some  boards  could  superimpose  the  computer 
signal  over  a  television  image  (by  synchronizing  or  genlocking  the  two  signals),  and 
output  an  RGB  or  composite  television  signal,  which  could  be  recorded  on 
videotape.  These  devices  are  intended  to  provide  sophisticated  television 
production  capabilities  to  a  video  non-professional  (thus  their  output  of  a  television 
signal,  and  the  term  "desktop  video"),  as  well  as  some  limited  incorporation  of 
television  images  into  a  computer /digital  format  (Wells,  1989).  The  most 
sophisticated  systems  (such  as  Newtek's  Video  Toaster)  provide  the  same 
capabilities  (albeit  of  lower  quality)  of  several  separate,  dedicated  machines  normally 
found  only  in  broadcast-quality  video  editing  facilities.  To  the  extent  that  these 
boards  produce  a  television  signal  as  output,  they  run  counter  to  the  prevailing 
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trend  of  multimedia  ,  which  is  to  incorporate  all  media  into  a  single,  digital  format 
for  display  on  a  single  monitor. 

In  all  of  these  previously  described  technologies,  the  video  signal  must  come  from  a 
conventional  analog  source  (such  as  a  VCR  or  a  videodisc  player)  and  tends  to  retain 
its  fundamental  identity  as  a  separate,  analog  signal  during  its  display  or  playback. 
This  approach  can  be  described  as  managed  video  (Arnett,  1990a  &  1990b),  in  which 
the  computer  serves  as  a  control  device  for  a  conventional  external  video  source. 
The  next  step  is  integrated  video,  in  which  motion  video  is  converted  to  digital 
form  and  then  is  treated  as  just  another  window  on  the  screen,  and  managed  in  the 
same  way  as  any  other  file  type,  like  a  graphics  or  text  file.  Such  video  can  be 
manipulated  (e.g.,  re-sized,  re-edited,  combined  with  other  file  formats)  with 
software  controls.  There  are  currently  two  approaches  to  integrating  (digitized) 
video  in  computer  systems.  Intel's  DVI,  which  has  been  under  development  for  the 
last  few  years,  requires  special  hardware  in  the  form  of  add-on  boards  to  process  and 
display  video.  On  the  other  hand,  software-only  solutions,  such  as  Apple's 
QuickTime  offer  greater  simplicity  of  use  and  would  seem  to  be  the  more  highly 
integrated.  Although  a  digitizing  board  is  required  to  create  QuickTime  "movies" 
(video  files),  they  can  be  played  back  on  any  Macintosh,  without  special  hardware. 
At  this  writing,  however,  DVI  provides  the  better  image  quality,  as  might  be 
expected  of  a  product  with  special  hardware  and  a  two  to  three  year  head  start, 

Problems  and  Pressures  of  Integration:  Digital  vs.  Analog  Video 
The  problem  in  integrating  video  into  the  computer  system  (in  addition  to  the 
signal/display  incompatibility  mentioned  previously)  is  the  large  amount  of 
information  in  a  television  signal,  combined  with  the  relative  slowness  of  the 
computer's  data  bus  (its  "bandwidth").  A  single  frame  of  raw,  uncompressed  video 
can  requires  as  much  as  one  megabyte  of  storage  space,  depending  on  the  computer 
system.  This  file  size  has  made  it  virtually  impossible  to  store  or  play  back  motion 
video  at  a  rate  approaching  "real  time"— that  is,  at  thirty  frames  per  second.  Even 
the  fastest  and  largest  hard  drives  cannot  access  and  write  such  large  files  to  the 
screen  at  this  rate.  For  this  reason,  many  developers  are  working  on  various  forms 
of  video  compression,  so  that  video  frames  can  be  stored  in  units  much  smaller 
than  1  MB.  Complex  algorithms  are  required  to  accomplish  video  compression,  but 
fortunately  two  standards  have  already  emerged.  JPEG  (Joint  Photographic  Expert's 
Group)  is  a  high  quality  standard  for  compressing  and  storing  single  frames  of  video 
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and  is  excellent  for  photorealistic  stills  such  as  slide  images,  but  it  requires  too  much 
dis!  space  and  bandwidth  for  motion  video  on  personal  computers.  On  the  other 
hand,  the  algorithms  used  by  MPEG  (Motion  Photographers  Expert's  Group),  for 
motion  video,  compares  information  between  successive  frames  so  that  similar 
information  is  not  repeated.  Although  MPEG  is  not  yet  an  official  standard,  many 
companies  are  working  feverishly  to  incorporate  these  standards  into  their  products 
(Arnett,  1990a  &  1990b).  For  example,  the  first  release  of  Quicktime  uses  JPEG  for  its 
still-frame  capture,  but  does  not  yet  achieve  the  MPEG  standard  for  motion.  On  the 
other  hand,  DVI  will  support  MPEG  by  late  1992. 

Why  is  there  pressure  to  convert  all  information  into  digital  form?  The  answer  is 
that  digital  data  is  the  common  denominator  for  all  computer  systems,  and 
therefore  vastly  simplifies  problems  of  storage,  playback,  transmission  (over 
networks)  and  display.  Similar  tools  can  be  used"  used  to  create,  edit  and  display 
video  as  are  currently  used  to  create  computer  graphics  or  to  edit  text  documents. 
When  video,  graphics,  text  and  sound  are  simply  different  computer  file  types, 
development,  delivery  and  display  becomes  vastly  simplified.  Storage  and  retrieval 
of  all  media  in  digital  form,  in  addition  to  a  technical  simplification,  promises 
integration  of  many  different  media  into  a  single  new  medium:  multimedia. 
Furthermore,  only  by  conversion  of  video  to  digital  form  can  it  be  made  available 
over  computer  networks. 


Technical  Factors:  A  Matter  of  Trade-Offs 
A  fundamental  premise  in  the  world  of  computer  interface  design  is  that  "There  are 
no  simple  answers,  only  trade-offs"  (Norman,  1987).  Trade-offs  can  be  defined  as 
interrelated  computer  attributes  that  tend  to  draw  on  the  same  limited  resources 
within  a  computer  system:  memory  size,  disk  space,  processing  speed,  transmission 
bandwidth,  etc.  For  the  multimedia  designer,  therefore,  it  is  in  fact  impossible  to 
freely  mix  media  in  courseware  due  to  the  inevitable  limits  on  processing  power 
and  memory  of  the  computer.  A  premise  of  this  article  is  that  developers  must 
learn  the  limits  imposed  by  the  technical  constraints  of  their  hardware  and  software, 
so  that  they  can  best  mix  and  match  media  to  kt  the  instructional  design  while  also 
following  the  most  economic  path  within  the  technical  constraints. 
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Norman  (1987)  has  developed  a  quantitative  comparison  method,  called  trade-off 
analysis,  which  adopts  the  power  function  from  psychological  research.  Briefly,  this 
method  computes  User  Satisfaction  values  for  various  attributes  of  the  computer 
interface  (i.e.,  disk  access  speed,  image  size,  image  quality,  etc.),  and  then  compares 
two  interface  variables  graphically  so  that  an  appropriate  compromise,  or  trade-off, 
can  be  selected.  For  example,  user  satisfaction  with  image  size  could  be  compared 
with  user  satisfaction  with  time  required  to  d^play  an  image  in  order  to  obtain  the 
most  appropriate  compromise,  because  improving  either  one  requires  substantial 
system  power.  In  addition  to  giving  confidence  and  justification  for  the  necessary 
compromises  in  design  decisions,  this  technique  may  show  that  a  satisfactory 
compromise  is  not  possible  for  different  classes  of  users,  such  as  experts  and  novices 
who  must  use  the  same  system.  The  major  criticism  of  this  method,  as  Norman 
(1987)  notes,  is  that  there  is  currently  little  or  no  quantitative  data  available  for  user 
satisfaction,  especially  for  variables  in  the  brave  new  world  of  multimedia  usage 
(e.g.,  what  are  the  acceptable  ranges  tor  motion  video  frame  rate,  image  size  or 
resolution?).   It  would  seem  that  acquiring  measures  of  user  satisfaction  with 
various  trade-off  parameters  for  multimedia  would  be  a  useful  and  fruitful  area  of 
research.  For  the  purposes  of  this  article  (and  in  keeping  with  the  philosophy  of  the 
tradeoff  analysis),  it  would  seem  sufficient  to  identify  and  isolate  the  key  factors 
involved  so  that  we  may  be  aware  of  them.  In  so  doing,  we  must  realize  that  the 
usefulness  and  satisfaction  with  the  entire  courseware  program /computer  system 
will  be  due  to  the  sum  of  its  parts  (Norman,  1987). 

Prototypical  Trade-Offs  for  Decision-making 

Before  considering  specific  technical  factors  that  fit  into  the  trade-off  formula,  it  will 
be  advantageous  to  consider  two  protoypical  trade-offs  that  have  recurred 
throughout  the  history  of  computing  and  are  even  more  evident  as  multimedia 
becomes  more  prevalent.  These  can  be  described  as  information  versus  time  and 
convenience  versus  quality.  These  two  trade-offs  can  provide  a  framework  for  the 
kinds  of  choices  that  must  be  made  when  designing  and  developing  multimedia 
courseware. 

One  of  the  more  onerous  aspects  of  working  with  computers  is  how  systems  slow 
down  as  a  greater  informational  burden  is  placed  upon  them.  A  simple  example 
concerns  the  integration  of  user-friendly  devices  such  as  menus,  windows,  and 
plentiful  help  screens.  Although  the  progressive  increase  in  computing  power  has 
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made  such  devices  a  standard  feature  of  personal  computers,  it  was  not  so  long  ago 
that  their  use  could  be  counted  on  to  slow  down  systems  considerably.  It  is  ironic 
that  the  more  user-friendly  a  computer  system  becomes,  the  slower  it  can  be 
expected  to  operate;  experts  eschewed  such  programmatic  helps  in  favor  of 
lightning-fast  responses,  which  required  memorizing  and  inputting  obscure, 
abbreviated  keyboard  commands.  Such  a  trade-off  is  most  noticeable  in  the  display 
of  increasingly  realistic  images.  Only  a  few  years  ago,  a  monochrome  diagram, 
composed  of  jagged  squares,  was  the  most  we  could  expect  in  the  way  of  graphics  on 
a  computer  screen;  today,  near-photographic  reproduction  quality  for  still  images, 
and  television-quality  motion  video,  are  becoming  the  standards  of  multimedia. 
Displaying  these  kinds  of  graphics,  however,  has  required  the  kind  of  storage  space 
and  horsepower  that  was  available  only  on  mainframes  a  decade  ago. 

Kay  (1990)  takes  this  notion  one  step  further  in  describing  a  trade-off  between  speed- 
and-power  vs.  exposition-and-thoughtfulness,  which  relates  to  the  use  of  artificial 
intelligence  in  interface  design.  Briefly,  it  may  be  preferable — yet  more  expensive  in 
terms  of  system  resources — to  have  a  system  that  can  adapt  to  the  user's  needs  and 
can  filter  incoming  data  in  order  to  supply  more  suitable  information,  rather  than 
simply  supply  more  information  at  a  higher  rate.  The  question  is  whether  we  want 
to  build  machines  that  are  modeled  more  as  intelligent  personal  servants,  or  as 
more  powerful,  faster  engines.  Again,  there  are  no  right  or  wrong  answers;  the 
correct  soution  depends  on  the  needs  cf  the  specific  situation.  In  the  example  at 
hand,  multi-media  will  obviously  require  more  system  resources  than  a  single 
medium.  Yet,  as  Bruner  and  Olsen  (1973)  have  noted,  multiple  media  may  be 
necessary  in  order  to  provide  the  variety  of  perspectives  that  different  learners,  with 
different  skills  and  abilities,  require  to  achieve  a  complete  undorstanding  of  the 
underlying  knowledge. 

The  great  boon  of  media— especially  multimedia — is  its  convenience.  As  every 
schoolchild  knows  by  now,  a  videodisc  is  capable  of  storing  54,000  single  images 
(which  can  be  played  back  at  30  images  per  second  to  serve  up  30  minutes  of  motion 
video),  which  is  the  equivalent  of  675  slide  trays,  each  holding  eighty  35mm  slides. 
This  is  a  staggering  statistic,  but  as  anyone  who  has  compared  the  image  quality  of  a 
35mm  slide  and  a  videodisc  image  (even  displayed  on  the  very  best  monitor) 
knows,  the  two  are  not  the  same.  Much  detail,  sharpness,  and  subtleties  of  color  are 
lost  due  to  the  limitations  of  the  video  signal.  If  we  extend  this  line  of  thought,  we 
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can  consider  that  the  slide  itself  is  a  cojy  of  a  real  scene  (or  a  work  of  art,  or  a  real 
object),  then  we  are  several  times  removed  from  the  direct  experience  of  the 
original.  We  may  be  able  to  store  reproductions  of  reproductions  of  54,000  works  of 
art  on  a  single  piece  of  plastic,  but  what  else  have  we  lost  by  merely  asking  students 
to  recognize  reality  rather  than  experience  that  reality?  The  result  of  this  trade-off, 
which  is  most  prevalent  in  computer-based  multimedia,  is  to  substitute 
representation  for  direct  experience.  The  trap  for  which  developers  must  constantly 
be  aware,  therefore,  is  not  merely  to  use  the  power  of  multimedia  to  present  more 
opportunities  for  representing  reality — which  is  easy — but  to  give  students  more 
opportunities  to  experience  reality  in  different  ways.  Simulations  are  a  step  toward 
this  direction,  but  the  challenge  is  use  multimedia  to  create  ever  more  powerful  and 
realistic  simulations.  In  order  to  counteract  this  tendency,  instructional  developers 
might  use  the  computer  as  an  auxiliary  tool — for  example,  to  interface  with  and 
analyze  reality — and  not  always  as  the  centerpiece  of  a  presentation  (Kay,  1991). 

Current  Technical  Trade-offs 

With  these  general  trade-offs  in  mind,  let  us  now  look  at  the  current  state  of 
constraints  and  see  where  we  ,:re  and  where  we  are  heading.  In  general,  a  set  of 
trade-offs  within  a  given  system  means  that  a  gain  in  one  of  these  areas  will  result 
in  a  loss  in  another  area,  if  all  other  system  resources  are  held  constant.  Remember, 
these  trade-offs  are  being  discussed  primarily  from  the  viewpoint  of  the  storage, 
processing  and  display  of  computer  (digital)  images.  Analog  video  devices  (such  as 
laser  videodisc  and  videotape)  have  similar/analogous  constraints,  but  the  technical 
specifications  for  analog  video  images  are  fixed  by  international  standards  and  are 
much  less  variable  than  digital  (e.g.,  30  frames  per  second,  512  horizontal  lines; 
vertical  lines  depend  on  the  quality  of  the  recording  medium  and  the  capture, 
transmission  and  display  devices). 

« 

•   Resolution  is  the  apparent  sharpness  of  the  image  as  it  appears  on  the  screen. 
Images  with  high  resolution  appear  sharper  and  crisper,  with  the  result  that  details 
are  more  visible  and  easily  inspected.  Resolution  is  affected  dramatically  at  each 
stage  of  the  recording  process,  starting  with  the  lens  quality  of  the  camera,  through 
the  quality  of  film  or  the  tape  format  used  for  recording  (Betacam  vs.  VHS),  the  type 
of  transfer  process  from  film  to  tape,  and  ending  with  the  final  display  medium.  In 
terms  of  program  content,  resolution  can  be  relatively  unimportant  (a  talking  head), 
or  extremely  important  (a  microscope  slide  of  plant  cells  or  an  x-ray  radiograph). 
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Resolution  is  a  more  subjective  quality  than  it  might  seem.  Experience  shows 
that  audiences  will  accept  lower  levels  of  image  resolution  unless  there  is  a  a  side- 
by-side  comparison  with  an  image  of  higher  resolution.  For  example,  we  know  that 
VHS  tape  has  the  worst  image  resolution  of  any  video  medium,  and  is  several 
orders  of  magnitude  worse  than  projected  35mm  motion  picture  film,  yet  audiences 
willingly  embrace  it  because  of  the  great  convenience  it  offers  (remember  the 
convenience  versus  quality  trade-off).  The  lesson  to  be  learned  here  is  to  be  careful 
about  mixing  similar  image  types  with  different  resolutions  on  the  same  screen. 
However,  side-by-side  screens — one  displaying  video  and  the  other  digital  data — 
could  most  likely  have  different  resolutions  if  the  audience  has  been  conditioned 
and  habituated  to  viewing  specific  types  of  information  on  each  screen.  This  issue 
of  ''frame  protocol"  (Hannafin  &  Rieber,  1989),  will  be  discussed  later. 

•   Image  size  is  a  constraint  in  digitized  images.  Larger  image  size  translates 
directly  to  more  pixels,  meaning  larger  file  sizes  and  CPU  processing  requirements. 
This  means,  for  motion  video,  that  larger  files  must  be  retrieved,  processed  and 
displayed  at  a  high  rate.  Some  early  versions  of  digitized  motion  video  (such  as 
Quicktime,  as  of  this  writing)  look  their  best  at  only  1/16  normal  screen  size. 
However,  this  postage-stamp  size  results  in  an  acceptable  apparent  sharpness, 
although  this  same  image,  if  enlarged,  would  be  seen  to  have  very  low  resolution. 
Again,  image  size  trade-offs  do  not  affect  analog  video  displays,  since  they  are 
limited  only  by  the  display  device  (the  size  of  the  monitor  screen  or  the  power  of  the 
video  projector). 

Image  size  constraints  will  drive  the  choice  and  design  of  images  displayed  on  the 
computer  screen.  If  a  small  image  is  acceptable,  then  it  should  be  used  in  the 
interest  of  conserving  system  resources.  For  example,  a  talking  head  or  a  close-up  of 
some  type  of  action  (such  as  hands  assembling  a  machine)  will  probably  work  well; 
however,  a  landscape  or  a  large  photomicrograph  with  significant  detailed  areas  will 
not  be  acceptable  in  a  thumbnail  size.  Videographers  who  record  scenes  for  use  in  a 
window  on  a  computer  screen  should  keep  the  small  size  constantly  in  mind,  just  as 
motion  picture  cinema tographers  who  made  the  transition  to  television  many  years 
ago  were  forced  to  change  their  habits  of  screen  composition. 
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•  Color  depth  refers  to  the  number  of  colors  that  can  be  shown  simultaneously  on 
the  screen  at  one  time.  The  usual  parlance  refers  to  the  size  of  the  digital  encoding 
unit  for  each  pixel  on  the  screen:  2-bit  color  allows  16  simultaneous  colors,  8-bit 
color  =  64  simultaneous  colors,  16-bit  ■  64,000  colors,  24-bit  =  16  million  colors;  32-bit 
images  allow  the  same  color  depth  as  24-bit  with  the  extra  8  bits  providing  accessory 
information.  The  trade-off,  as  in  image  size  and  resolution,  is  simple:  more  colors 
equals  more  data  and  a  larger  file  size.  Therefore,  designers  should  use  the  smallest 
color  depth  (fewest  number  of  colors)  that  is  workable.  For  example,  a  talking  head 
should  not  require  24-bit  color,  unless  facial  details  are  important.  However,  an  art 
history  expert  may  find  even  24-bit  color  somewhat  constraining,  depending  on  the 
resolution  of  the  monitor. 

•  Speed  refers  to  the  rate  at  which  new  frames  can  be  displayed  on  the  screen,  that 
is,  the  refresh  rate  and  the  frame  rate.  Once  again,  speed  is  not  an  issue  with  analog 
devices,  such  as  videodisc  or  videotape.  True  motion  video  requires  thirty  new 
frames  to  be  written  on  the  screen  every  second  (NTSC  standard).  Standard 
QuickTime,  for  comparison  purposes,  can  only  manage  about  twelve  frames  per 
second,  and  with  other  trade-offs,  the  range  can  go  as  high  a  15  or  as  low  as  eight  or 
nine  per  second.  DVI,  on  the  other  hand,  normally  plays  at  a  full  30  frames  per 
second,  as  befits  a  hardware-based  system. 

When  video  is  recorded  at  30  fps  and  must  be  played  back  at  a  reduced  rate  due  to 
system  constraints,  two  things  can  happen.  If  all  frames  are  displayed,  the  action 
appears  in  slow  motion  and  the  result  is  not  "real  time";  or  the  computer  skips 
frames  in  order  to  maintain  "real  time",  the  results  appear  jerky,  especially  if  the 
frames  are  skipped  intermittently.  How  does  this  choice  affect  design  decisions?  A 
talking  head  must  be  played  back  in  real  time  in  order  to  match  the  audio,  but 
watching  someone  talk  jerkily  at  10  fps  is  rather  disturbing.  A  designer  would 
probably  put  speed  of  playback  as  a  top  priority  for  a  talking  head  and  sacrifice  image 
quality  and  size.  On  the  other  hand,  most  graphic  animations  do  not  require 
playback  at  30  fps.  For  purely  economic  reasons,  many  broadcast  animated  cartoons 
are  in  fact  recorded  as  two  (or  three)  frames  of  video  or  film  for  each  graphic  frame, 
which  gives  them  an  effective  speed  of  15  (or  10)  frames  for  every  second.  Because 
these  cartoons  are  so  packed  with  action,  our  eyes  tend  to  fill  in  the  blanks.  In  this 
case,  a  slow  speed  is  acceptable  and  resources  can  be  re-allocated  to  increase  color  and 
resolution. 
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•  Data  Rate  and  Bandwidth— Data  rate  refers  to  the  speed  at  which  data  is  read 
from  the  storage  medium,  while  bandvfidth  is  a  related  factor  that  refers  to  the 
carrying  capacity  of  the  transmission  channel  used  to  transfer  the  data,  both 
parameters  are  therefore  related  to,  but  different  from,  the  speed  factor  just 
described.  For  example,  CD-ROM's  are  notoriously  slow  at  reading  data,  and  for  this 
reason  cannot  currently  be  used  for  direct  playback  of  uncompressedvideo,  despite 
their  tremendous  storage  capabilities.  Networks  also  tend  to  be  bottlenecks  for 
transmission  of  large  files,  such  as  video,  at  a  fast  enough  rate  for  suitable  display  of 
motion. 

Summary 

When  choosing  a  display  format  for  interactive  multimedia,  the  decision  about  how 
and  whether  to  mix  analog  and  digital  images  is  fundamental,  as  is  the  decision 
about  whether  to  use  managed  or  integrated  video.  Managed  video  is  the  current 
norm,  and  the  simplest  method  is  to  keep  the  analog  display  separate  from  the 
digital  display.  An  added  level  of  complexity  and  cost  is  required  to  overlay  the 
computer  signal  on  the  video  display.  True  integrated  video  requires  conversion  of 
the  analog  source  to  digital,  and  adequate  storage  and  processing  power  for  retrieval 
and  display.  Once  the  important  technical  constraints  of  any  computer-based 
multimedia  system  are  understood  and  delineated  as  interrelated  trade-offs,  it  will 
be  easier  to  make  decisions  regarding  the  components  of  computer  system.  It  may 
be  helpful  to  remind  ourselves  that  most  system  constraints  tend  to  level  off  as  the 
technology  advances  (e.g.,  storage  capacities,  processing  power,  transmission 
bandwidth).  All  trade-offs,  except  for  the  prototypical  trade-offs  that  tend  to  be 
endemic  to  computer  systems  in  general,  tend  to  become  less  constrictive  as  time 
goes  on. 

Practical  ConsideraHons 

System  requirements  and  constraints,  and  the  theoretical  factors  that  affect  learning, 
are  important  matters  that  should  concern  anyone  who  has  the  responsibility  for 
designing  and  developing  a  multimedia  display  system.  The  following  section  is 
intended  to  address  the  more  practical  matters  that  will  determine  the  actual 
configuration  of  a  workstation  or  a  group  of  workstations  in  a  particular  context 
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.  Some  of  these  considerations  are  direct  implications  of  previously-described 
technical  and  theoretical  factors. 

1.  An  single  screen  system  that  incorporates  analog  video  with  computer  overlay 
is  typically  more  technically  complex  (and  therefore  more  expensive)  than  a  two- 
screen  system,  despite  the  requirement  foi*  an  extra,  video-only  monitor. 
However,  single-screen  systems  that  use  integrated  (digital)  video  can  be  much 
simpler,  except  that  they  now  require  more  powerful  computers  and  very  large 
storage  capacities  (i.e.,  hard  disks  in  the  gigabyte  range,  CD-ROM  drives,  high- 
capacity  tape  drives),  or  networks  with  high-bandwidlh  capabilities  (e.g., 
Ethernet). 

2.  Two-screen  systems  are  simpler  to  control  from  a  programming  standpoint  than 
single-screen  managed-video  i  /stems.  For  example,  a  Macintosh  two-screen 
system  requires  only  a  single  cable  and  minimal  software  (e.g.,  HyperCard)  in 
order  to  provide  complete  control  of  a  videodisc  player.  However,  integrated 
systems  show  promise  of  also  being  very  easy  to  control;  for  example,  HyperCard 
can  be  used  to  control  Quicktime  movies  with  a  minimum  of  complexity. 

3.  Altering  single  screen,  managed-video  courseware  programs  (computer  graphics 
overlaid  on  video)  will  probably  be  more  problematic  than  altering  programs 
that  have  kept  the  two  media  separate  (dual  screen).  However,  single  screen 
integrated-video  designs  will  ultimately  offer  greater  flexibility  for  modification. 

4.  Two-screen  systems  will  require  more  physical  space  (for  the  second  monitor,  as 
well  as  for  the  video  source  device,  probably  a  videodisc  player). 

5.  Two  screen  systems  have  twice  as  much  physical  display  (screen)  space  for 
presenting  information.  They  can  present  different  levels  of  image  quality 
without  disturbing  the  viewer,  as  would  be  the  case  if  different  quality  levels  are 
mixed  on  a  single  screen. 

6.  Two-screen  systems  require  observing  a  more  complex  and  uniquely  defined 
frame  protocol,  which  is  defined  as  the  "systematic  use  of  available  screen  space 
for  defined  purposes"  (Hannafin  &  Rieber,  1989).  Also,  a  cardinal  principle  of 
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interface  design  is  consistency  (Rubinstein  &  Ilersh,  1984),  and  a  second  screen 
compounds  the  problem  of  consistency  of  use. 

7.  Two  screen  systems  may  require  stronger  and  more  frequent  and  elaborate  cuing, 
so  that  students  will  know  when  and  where  to  direct  their  attention  to  the 
second  screen. 

8.  Two  screen  systems  will  offer  new  opportunities  and  greater  flexibility  for  varied 
or  learner-control  system  uses.  For  example,  the  video  screen  could  be  used  as  a 
television  monitor,  or  as  a  stand-alone  monitor  for  a  hand-controlled  (or  bar- 
code controlled)  videodisc  player.  Likewise,  depending  on  the  capabilities  of  the 
computer  system,  the  computer  could  be  used  alone  with  the  same  capabilities  as 
a  single-screen,  all-digital  multimedia  platform. 

The  decision  about  which  hardware  configuration  to  purchase  will  depend  on  a 
combination  of  factors,  based  upon  theoretical,  technical  and  practical 
considerations.  It  is  to  be  expected  that  no  system  will  be  a  perfect  match,  and  that 
compromises  must  be  made.  Remember  that  "there  are  no  simple  answers,  only 
trade-offs"  (Norman,  1987). 


Decisions  Affecting  the  Plant  Biology  Multimedia  Workstation: 

A  Case  Study  Outline 

Recently,  the  author  and  his  colleagues  were  required  to  make  decisions  about  the 
hardware  for  multimedia  student  workstations  to  be  used  in  a  sophomore-level 
plant  biology  laboratory.  The  following  is  an  outline  of  the  requirements  prioritized 
into  two  categories,  based  on  course  objectives. 

Important  requirements: 

•  Highest  possible  resolution,  color  rendition,  and  image  size  for  graphic  and 
photographic  images 

•  Storage  for  a  large  number  of  images  (photographs,  animations,  still  pictures,  etc.) 

•  Create  and  maintain  a  database  of  images  that  could  be  accessed  apart  from  the 
computer  system  (i.e.,  with  hand  controller  or  bar  code,  as  a  classroom  display  or  for 
individual  use  in  a  media  carrel) 
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•  Moderate  need  for  real-time  motion  video 

•  Fairly  low-cost  system 

•  User-friendly,  easy-to-learn  system 

•  High  system  speed  (no  long  waits  for  system  to  write  a  graphic  to  screen  or 
respond  to  user  input) 

•  Variety  of  software:  simple  database  of  images;  classic  self-instructional  tutorials; 
several  simulations;  some  computer-based  tools,  such  as  cladistic  programs,  word 
processors,  etc. 

•  Flexibility  to  use  a  variety  of  different  courseware  at  various  levels  of 
interactivity,  some  of  which  has  not  been  identified  (e.g.,  Level  1  and  2  videodiscs; 
computer  tools,  such  as  word  processors  and  various  scientific  programs) 

Less  important  requirements: 

•  User  manipulation  of  realistic  (video)  images 

•  Complete  integration  of  text  and  images  (i.e.,  images  can  be  satisfactorially  used  as 
illustrations  of  text) 

•  Conservation  of  physical  space  (i.e.,  extra  equipment  is  not  a  problem) 

•  Maintaining  of  all  information  in  digital  format 

•  Frequent  updates  of  visual  image  bank 

System  Configuration 

The  final  decision  favored  a  classic  two-screen  system,  in  which  the  computer 
(Macintosh  Ilsi  with  a  high-resolution  monitor  )controls  the  videodisc  player 
(Pioneer  2200  with  a  high-resolution  video  monitor).  Although  there  is  a  relatively 
high  cost  involved  with  updating  a  visual  database  that  is  stored  on  videodisc, 
image  quality  requirements,  system  flexibility  and  the  current  flux  and  uncertainty 
in  digital  video  presentation  and  storage  dictated  the  choice  of  this  design. 

Coincidentally,  this  dual  screen  arrangement  matched  certain  skill  patterns  that 
students  of  biology  must  develop:  finding  and  matching  details  on  diagrams  to 
realia,  or  real-life  objects  (i.e.,  matching  parts  of  a  diagram  of  a  tissue  to  a  section  of 
tissue  on  a  microscope  slide  or  a  photomicrograph).  This  carry-over  of  skill  patterns 
helped  us  to  specify  a  consistent  design  feature,  or  screen  protocol,  that  we  tried  to 
maintain  throughout  the  program:  restrict  the  computer  screen  to  symbolic 
information  (text,  diagrams,  etc.)  and  the  video  monitor  to  realistic  images  and 
highly  detailed  graphics. 
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On  the  surface,  the  decision  in  favor  of  a  two-screen  system  may  seem  to  run 
counter  to  the  trend  toward  single  integrated  systems;  yet  it  maintained  the  required 
system  flexibility  and  freed  the  computer  to  perform  those  things  that  a  computer 
does  well.  It  also  maintained  an  acceptable  system  speed  for  a  relatively  low-cost 
system  (Macintosh  Ilsi)  with  a  user-friendly  interface,  and  yet  allowed  for  a  high 
degree  of  interactivity. 
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CURRICULUM  MATERIALS  CENTER 
IN  COLLEGES  OF  EDUCATION 


Background 

The  curriculum  material  center  (CMC)  is  an  area  typically  located  in  a  college  of 
education  building.  Such  a  center  usually  provides  a  full  range  of  printed  and  non- 
printed  materials,  equipment,  and  delivery  of  support  services  to  faculty,  students,  and 
teacher  education  programs  in  a  college  of  education. 

The  primary  mission  of  the  CMCs  involves  many  processes  in  respect  to  what 
students  and  teachers  do  with  materials  evaluation,  synthesis,  reflection  thinking, 
appreciation,  and  assembling  of  materials  (Ward  &  Beacon,  1973,  p.  31).  Albright 
(1989)  pointed  out  that  an  additional  mission  of  a  curriculum  material  center  is  "to  help 
college  faculty  members  improve  their  teaching/*  In  addition,  they  need  help  in  such 
activities  as  "course  planning,  discussion,  diagnosing  student  misunderstandings  of 
course  content,  writing  tests,  and  course  evaluation"  (p.  41).  The  CMC  is  valuable  to 
teacher  education  programs  as  they  function  to  select,  house,  and  make  available  new 
materials.  According  to  Vlcek  &  Wilman  (1989),  the  basic  purpose  of  curriculum 
material  centers  is  "to  develop  teaching/learning  strategies,  programs,  and  materials 
that  facilitate  learning  of  training  performance"  (p.  11).  The  personnel  serving  in  the 
CMCs  ought  to  provide  professional  directions  and  guidance  to  center  clients. 
Curriculum  material  center  personnel  ought  to  be  trained  to  keep  apace  with 
advancement  of  the  modern  technology.  Teacher  education  institutions  must  support 
their  CMCs  "so  that  the  quality  of  their  service  will  enhance  the  over  all  teacher 
education  programs"  (Ellis,  1969,  pp.  31). 

Curriculum  material  centers  exist  to  facilitate  and  improve  learning  by  supporting 
classroom  instruction  as  well  as  supporting  services  to  student  teachers  staff,  faculty, 
and  the  community.  The  curriculum  materials  center  ought  to  be  an  organization  that  is 
charged  with  the  responsibility  of  facilitating  the  continuing  development  of 
professional  and  personal  competencies  of  faculty,  particularly  those  that  lead  to  the 
improvement  of  teaching  and  learning  (Albright  1983;  Gaff,  1975). 

The  Statement  of  fcfcs  Problem 
There  are  justified  demands  for  current  information  concerning  organizational 
structure,  management  processes,  funding,  and  }>erceptions  related  to  the  relation  of 
CMC's  as  it  functions  and  value  in  the  preparation  of  t?achcr  education  programs.  This 
study  attempted  to  identify  the  relationship  between  the  existence  of  the  CMC's  and 
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the  effects  of  accreditation  and  standards  upon  CMC's,  and  the  factor  which  represents 
the  greatest  barrier  to  the  efficient  and  effective  integration  of  the  CMC's  into  the 
respective  teacher  education  programs,  This  study  is  limited  to  103  colleges  of 
education  in  selected  institutions  in  nineteen  (19)  states  representing  the  membership  of 
the  North  Central  Association  (NCA).  They  are  Arizona,  Arkansas,  Colorado, 
Illinois,  Indiana,  Iowa,  Kansas,  Michigan,  Minnesota,  Missouri,  Nebraska,  New 
Mexico,  North  Dakota,  Ohio,  Oklahoma,  South  Dakota,  Wesc  Virginia,  Wisconsin, 
and  Wyoming. 

This  study  compared  and  analyzed  the  perceptions  of  the  CMC 
coordinator/directors  and  the  administrator/dean  regarding  the  functions  and  value  of 
the  CMC's  in  the  context  of  the  present  and  the  future.  The  study  was  designed  to 
produce  a  profile  of  curriculum  material  centers  and  their  relation  to  the  organization 
of  colleges  of  education  and  the  respective  teacher  education  programs.  The  thrust  of 
the  study  was  1)  to  determine  the  purpose  of  the  centers,  2)  to  determine  the  impact  of 
these  centers  upon  the  teacher  education  programs  within  the  colleges,  3)  to  identify 
patterns  or  profiles  of  the  organization  and  management  of  the  centers,  and  4)  to 
determine  whether  the  CMCs  are  maintained  because  of  political  tradition,  i.e.,  to 
meet  the  requirements  of  external  agents  such  as  NCATE,  or  are  these  units  necessary  to 
the  effective  preparation  of  future  classroom  teachers  at  the  teacher  training  level. 
The  basis  of  these  four  concerns  were  set  within  a  view  of  perceptions  of  the  present 
context  versus  future  status  of  the  CMC's  in  the  colleges  participating  in  the  study. 

Significance  of  the  Study 

It  was  anticipated  that  the  results  of  the  research  would  provide  current  insight  as 
well  as  extensive  information  into  the  nature  and  the  "status  quo"  of  CMC's,  as  well  as 
probable  futures  for  such  centers.  This  study  would;  1)  provide  pertinent  information  to 
educational  administrators  for  use  in  planning  ?  nd  managing  more  efficient  and 
effective  CMC's,  in  an  administrative  component  of  colleges  of  education  in  the  present 
and  the  future;  2)  obtain  data  regarding  staff,  media  services,  facilities,  the  scope  of 
print,  non-print  and  equipment  collections  in  the  present  and  future  context;  3) 
determine  strategies  for  developing  instructional  media  support  via  the  CMC  as  a 
primary  and  integrated  factor  in  the  implementation  of  teacher  education  programs; 
and  4)  provide  a  profile  of  the  effects  of  NCATE,  as  those  standards  relate  to  the 
continued  acceptance  and  existence  of  CMC's  within  colleges  of  education;  5)  the 
development  oi  a  media  management  model  which  should  promote  a  more  efficient  and 
effective  planning  process  and  integration  of  CMC's  in  teacher  education  programs, 
including  pre-service  and  in-service. 

The  results  of  the  study  is  significant  in  that  it  brings  together  a  number  of 
administrative  and  organizational  structures  that  were  found  in  the  CMC's  from  which 
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the  data  were  obtained.  These  patterns  u,uld  be  used  for  the  justification  of  CMC's 
status  of  staff  and  personnel,  facilities  and  services,  and  the  selections  of  material 
holdings,  equipment,  and  administrative  control,  for  the  present  as  well  as  the  future. 

Design  of  the  Study 

Thirteen  hypothesis  were  developed  from  four  research  questions.  In  addition, 
hypotheses  were  formulated  to  determine  whether  the  justification  for  the  curriculum 
materials  centers  was  to  meet  accreditation  requirements  or  to  facilitate  and  improve 
K-12  teacher  preparation.  Thei  were  no  ready-made  questionnaires  suitable  for  this 
study.  As  a  result,  the  instrument  used  for  this  research  was  self-designed 
questionnaire. 

A  27-item  questionnaire  for  the  CMC  coordinators/directors  and  a  15-item 
questionnaire  for  the  college  executives  was  developed.  Comparison  of  mean  scores,  t- 
scores  and  pearson  correlation  coefficients  treatments  were  used  for  the  analysis  of  the 
data. 

After  considering  various  methodologies  which  could  be  used  to  meet  the  study 
objectives,  the  survey  technique  was  selected  for  use  in  this  study.  The  survey  technique 
was  employed  to  gather  data  concerning  the  current  and  future  status  of  the  CMC.  Two 
types  of  instruments  were  employed;  descriptive  methodology  and  a  Likert  scale 
format.  The  descriptive  survey  technique  was  employed  to  gather  data  concerning  the 
existing  status  of  the  CMC's.  The  Likert  scale  format  was  selected  for  the  survey 
instrument  because  it  is  reliable  and  can  be  suitable  to  people  with  a  particular 
attitude.  The  items  of  the  questionnaire  were  rated  on  a  scale  of  one  to  five(l-  5),  with 
designations  such  as  "strongly  disagree,  disagree,  undecided,  agree,  to  strongly  agree." 


Population  and  Sample 
The  universities  and  colleges  considered  for  this  study  were  selected  institutions 
which  prepare  elementary,  middle  and  secondary  school  teacher  candidates  in  19 
states  representing  the  membership  of  the  North  Central  Association.  All  together, 
103  institutions  were  selected.  The  target  population  of  this  study  consisted  of  four 
groups:  the  CMC  coordinator,  the  college  executive  of  large  colleges  with  an  enrollment 
of  10,000  or  more,  and  smaller  colleges  with  an  enrollment  between  3,000  to  9,999.  The 
population  was  composed  of  206  participants,  103  CMC  coordinators  and  103  college 
executives.  The  findings  of  this  study  were  based  upon  81  percent  rate  of  responses  from 
the  CMC's  college  executives  and  79  percent  from  coordinators. 

Analysis  of  Data 

After  the  data  was  collected,  all  items  were  prepared  and  arranged  lor 
examination.  The  t-test  treatment  was  used  to  compute  the  difference  between  the  two 
means  (Norusis,  1987,  stated  that  such  an  analysis  is  appropriate  for  such  data). 
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While  Pearson's  correlation  coefficient  was  chosen  due  to  the  equal  interval  nature  of 
the  data,  Hinkle,  Wiersma,  and  Jurs  (1979)  stated  that  Linear  regression  analysis  was 
appropriate  for  such  data.  In  addition,  descriptive  statistics  were  presented  due  to  the 
nominal  nature  of  the  responses.  Hinkle,  Wiersma,  and  Jurs  (1979)  stated  that  such  an 
analysis  is  appropriate  for  such  data).  SPSS-X  for  IBM/CMS,  at  the  University  of 
Toledo  was  used  to  analyze  the  data. 

The  Results  of  the  Study 

Hypotheses  were  formulated  to  determine  whether  the  justification  for  curriculum 
materials  centers  (CMC)  was  to  meet  accreditation  requirements  or  to  facilitate  and 
improve  k-12  teacher  preparation.  The  results  revealed  that  college  executives  differ 
in  their  perceptions  in  regards  to  the  standards  and  requirements  of  NCATE.  However, 
as  the  results  of  this  study  demonstrated,  the  college  executives  very  strongly  believed 
that  CMC's  exists  in  response  to  the  pressure  to  meet  the  needs  of  pre-service  teacher 
preparation  programs.  On  the  other  hand,  CMC  coordinators  very  strongly  believed 
that  CMCs  exist  to  meet  the  standards  and  requirements  of  NCATE. 

Further,  the  college  executives  and  the  coordinators  differ  in  their  perceptions  in 
regards  to  the  instructional  support  and  budget.  The  college  executives  suggest  that  the 
CMCs  were  not  provided  with  adequate  funding  to  carry  out  their  mission  in  the 
present.  However,  the  college  executives  predicted  that  the  CMC  of  the  future  will  be 
provided  with  appropriate  funding  and  instructional  support  to  carry  out  the  mission  of 
this  support  center.  The  college  executives  predicated  that  the  CMC  of  the  future  will 
be  provided  with  appropriate  funding  and  instructional  support  to  carry  out  the  mission 
of  this  support  center.  The  coordinators,  on  the  other  hand,  were  pessimistic  about 
funding  and  instructional  support  in  the  present  and  the  future.  However,  the  data 
indicated  the  CMC  coordinators  of  the  smaller  institutions  were  presently  doing  well  in 
resptv*  to  instructional  support.  Lastly,  both  college  executives  and  CMC  coordinators 
agreed  that  the  CMC  should  be  evaluated  in  terms  of  the  center's  impact  upon  the 
success  of  the  respective  teacher  education  programs  in  the  colleges  of  education,  as 
opposed  to  traditional  library  circulation  evaluation  models. 

The  descriptive  profiles  of  the  CMCs  in  both  small  and  large  institutions  of  higher 
education  produced  the  following  information  highlights: 

1 .  The  lines  of  authority  were  controlled  by  the  college  of  education. 

2.  The  staff  and  personnel  in  the  curriculum  center  ranged  from  one  or  two 
professional  staff  members,  and  from  one  to  two  support  staff  personnel. 
The  number  part-time  student  assistants  ranged  from  8  to  10. 

3.  Generally,  the  materials  and  equipment  collection  of  the  CMC  was 
purchased  by  the  college  of  education. 
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4.  Curriculum  materials  centers  were  budgeted  as  part  of  the  college  of 
education  or  department  of  education. 

5.  A  clear  majority  of  the  management  operations  in  the  large 
institutions  were  generally  computerized,  while  fewer  management 
operations  in  smaller  institutions  were  computerized. 

6.  A  high  majority  of  the  large  institutions  provided  pre-service 
teacher  education  classes/ workshops  on  a  regularly  scheduled  basis  in 
the  curriculum  material  center.  Yet,  only  a  small  portion  the  above 
services  were  provided  to  the  in-service  teachers.  However,  a  clear 
majority  of  the  smaller  institutions  provided  the  noted  services  to  both 
pre-service  teachers  and  in-service  teachers  (see  Table  1  &2), 

7.  Statistically,  the  major  clients  of  the  CMC  were  undergraduate 
students  in  teacher  education.  The  other  identified  clients  were 
graduate  students  and  faculty  in  the  college  of  education. 

8.  The  following  services  were  provided  to  the  clients  of  the  CMC: 
production  of  inexpensive  classroom  instructional  materials, 
video  (facilities/equipment,  microteaching  and  previewing); 
computers  (microcomputers)  for  client  use;  and  at  a  lower  level, 
photographic  services, 

9.  Curriculum  material  centers  maintained  standard  collections  of 
traditional  A-V  equipment,  textbooks  (K-12),  periodicals  (K-12), 
periodicals  (teacher  education),  and  curriculum  guides  (K-12), 

TABLE  1 

Pre-service  Teacher  Education  Classes/Workshops 
Regularly  Scheduled  in  the  Center 


Small  Institutions        Large  Institutions        All  Institutions 
Percent  Percent  Percent 


Are  Pre-service  teacher  education  CI  asses /workshops  regularly  scheduled  and  held  in 
the  CMC  using  the  facilities,  equipment  and  services  of  the  Center, 

Yes  63.6  80.4  73.4 

No  36.4  19.6  26.6 
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TABLE  2 

ln-tervice  Teacher  Education  Classes/ Workshops 
Regularly  Scheduled  in  the  Center 


Small  Institutions        Large  institutions      All  Institutions 
Percent  Percent  Percent 


Are  In-service  teacher  education  Classes /workshops  regularly  scheduled  and  held  in 
the  CMC  using  the  facilities,  equipment  and  services  of  the  Center. 

Yes  54.5  39.1  45.6 

No  45.5  58.7  53.2 

No  Response  0.0  1.0  1.3 


In  determining  the  significant  barriers  to  change  facing  institutional  planners 
related  to  the  future  of  CMCs,  the  study  found  positive  correlation  between  college 
executives  and  coordinators.  Table  one  lists  the  perceptions  of  the  college  executives 
and  CMC  coordinators  as  related  to  the  factors  representing  the  greatest  barriers.  The 
ranking  order  of  the  barriers  were  as  follows  (1  =  the  greatest  barrier  and  9  =  the  least 
difficult  barrier).  Clearly,  the  most  difficult  problem  for  CMCs  in  the  future  is  to 
obtain  and  retain  administrative  support  to  continue  their  existence. 

TABLE  3 

The  mean  Perceptions  of  the  College  Executives  and  CMC  Coordinators  as 
Related  to  the  Factors  Representing  the  Greatest  Barrier. 


Factors  With  the  Greatest  Barrier  Rank 


Space  3 

Qualified  staff  (Professional  and  Support)  5 

Leadership  by  Coordinator/Director  8 

Acceptance  by  students  9 

Institutional  Administrative  Support  1 

Faculty  involvement  2 

Organizational  structure  6 

Impact  of  new  technologies  of  instruction  7 

Adoption  of  new  technologies  of  instruction  4 
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Implications 

Curriculum  materials  center  directors  must  collect  better  data  to  identify  and 
present  the  CMC  needs  to  institutional  decision  makers.  Given  the  current  view  of  the 
future  trends  for  curriculum  material  centers,  the  following  implications  are  presented: 

•  The  primary  concern  is  the  lack  of  communication  between  the  college  executive  and 
the  curriculum  material  center  coordinators.  Regular  meetings  between  deans  and  CMC 
directors  must  take  place  to  promote  better  communication  and  change.  In  addition,  the 
appointment  of  curriculum  material  center  directors  to  deans1  administrative  decision- 
making groups  may  improve  communications  and  institutional  planning. 

•  The  management  plans  and  operational  services  of  the  CMC  should  be  reviewed  by 
an  advisory  committee  which  ought  to  include  faculty  representatives  from  each  of  the 
respective  teacher  education  areas.  When  such  a  representative  group  helps  establish 
priorities,  it  makes  it  possible  for  the  curriculum  material  center  coordinator  to  plan 
and  implement  suggestions  originating  from  within  the  programs  of  the  department  or 
the  college. 

•  It  appears  to  be  reasonable  and  necessary  that  colleges  of  education  ought  to 
provide  pr ^-service  and  in-service  teachers  a  wide  range  of  training  experiences  and  to 
introduce  them  to  the  use  of  all  types  of  instructional  materials  and  new  technologies  of 
instruction. 

•  As  curriculum  material  centers  grow  and  accommodate  more  pre-service  and  in- 
service  teachers  as  well  as  faculty  of  the  college  of  education,  it  will  become 
increasingly  more  difficult  to  justify  needs,  establish  policy,  budget,  and  manage  staff 
without  goals,  objectives  and  criteria  by  which  to  measure  or  evaluate  the  performance 
of  this  organizational  unit.  Rationally,  it  seems  that  what  is  needed  is  for  CMCs  to 
move  from  traditional  circulation  based  evaluation  to  models  which  are  more 
programmatically  based. 

•  Given  the  perceived  barriers  to  change,  deans  and  directors  must  work  together  on 
strategic  planning  in  order  to  guarantee  the  integration  of  the  curriculum  material 
center  into  the  future  of  the  college. 

•  In  comparing  the  descriptive  profiles  reviewed  in  Teclehaimanot  (1990)  and  Ellis 
(1969),  the  following  factors  remain  a  problem  for  the  curriculum  material  center: 

a.  the  organizational  structure  of  curriculum  material  centers  has  not  changed  to 
reflect  the  needs  of  modern  institutions  of  higher  education. 

b.  modern  CMCs  have  been  highly  affected  by  new  technologies  of  instruction. 

c.  aspec  ts  of  the  management  of  current  curriculum  material  centers  are  being 
computerized  at  various  levels  of  sophistication. 

d.  adequate  funding  and  institutional  support  remains  a  major  problem  for  CMCs. 

e.  the  leaders  of  curriculum  materials  center  in  the  1990s  must  be  pro-active  and 
seek  the  funds  to  purchase  equipment  and  materials  rather  than  relying  upon 
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donations  by  producers  and  publishers. 
The  major  findings  of  current  research  strongly  suggest  that  the  curriculum  material 
center  coordinators  and  the  college  executives  (deans  or  designers)  need  to  collaborate, 
plan  and  clarify  their  respective  values  related  to  the  CMC  and  NCATE  standards. 
Further,  different  perceptions  regarding  the  nature  and  function  of  curriculum  material 
centers  continue  to  exist. 
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Introduction 


For  the  last  two  years,  we  have  been  developing  computer-based  business  simulations  to  help 
prepare  high  school  students  for  entry-level  jobs  in  business  settings.  In  designing  our 
simulations,  we  wanted  to  augment  ihe  simulated  environment  with  additional  instructional 
supports  to  bolster  and  focus  student  learning.  Feedback  mechanisms  were  seen  as  appropriate 
pedagogical  tools,  but  ones  that  had  to  be  used  carefully  to  avoid  destroying  the  seductive  and 
instructional  power  of  the  simulation  fiction.  Unfortunately,  the  research  literature  was  largely 
silent  about  the  use  of  feedback  mechanisms  for  simulations.  The  feedback  mechanisms  we 
developed  were  based  on  three  principles.  First,  feedback  mechanisms  should  support  the  user's 
intuitions  about  being  a  part  of  the  simulation  fiction.  Second,  feedback  mechanisms  should 
clarify  misunderstandings  and  str*  igthen  previous  learning.  Third,  feedback  mechanisms  should 
help  the  learner  generalize  to  similar  situations.  This  paper  explores  the  rationale  and 
implementation  of  our  feedback  mechanisms  to  help  other  simulation  designers  think  about  these 
issues  in  their  designs  and  to  generate  ideas  to  help  form  a  research  agenda  on  simulation  feedback. 


Why  Simulations? 

As  any  educational  technologist  knows,  different  instructional  media  are  appropriate  for  different 
instructional  needs.  In  preparing  high-school  students  for  entry-level  jobs,  we  choose  to  utilize 
simulations  because  we  felt  that  simulations  would: 

1 .  Provide  a  Learning  Context 

Higfr  school  students  get  very  little  exposure  to  realistic  business  settings.  By 
providing  them  with  experience  in  a  simulated  business  setting,  students 
would  gain  an  experiential  understanding  of  that  world. 

2.  Integrate  Disparate  Topic  Areas 

One  of  tiie  major  weaknesses  of  the  traditional  classroom  is  that  topic  areas 
become  artificially  separated.  For  example,  marketing  and  accounting  are 
taught  in  separate  classes.  Because  real -world  businesses  have  to  deal  with 
many  issues  simultaneously,  good  business  simulations  force  students  to 
integrate  different  topic  areas  in  realistic  ways. 

3.  Demonstrate  the  Realistic  Contingencies  of  the  Workplace. 

At  a  minimum,  simulations  provide  a  realistic  view  of  the  cause  and  effect 
relationships  in  a  particular  environment.  So  for  example,  users  of  a  business 
simulation  will  learn  that  if  customers  are  treated  poorly,  revenues  and  profits 
will  drop. 

4.  Motivate  Students  to  Learn  Basic  Skills 

By  being  introduced  to  a  realistic  workplace,  students  will  become  more 
motivated  to  learn  basic  skills  that  are  valued  in  the  workplace  (i.e.,  math, 
writing,  presentation). 

5.  Show  How  Every  Employee  is  Important  to  Business  Success 

By  getting  a  feel  for  how  different  functional  areas  must  work  together  to 
make  a  business  organization  a  success,  students  will  learn  how  each  job  and 
each  employee  is  vital  to  the  health  of  any  organization  they  might  join. 
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Empower  Students 


By  putting  the  students  in  the  role  of  the  general  manager  and  giving  them 
decision-making  authority  and  responsibility,  students  become  energized  to 
take  control  of  their  own  careers. 

7.   Elicit  Critical  and  Creative  Problem-Solving  Skills 

By  practicing  decision-making  and  seeing  how  different  decisions  have 
different,  yet  predictable,  results,  students  learn  the  importance  of  critical 
analysis  and  creative  problem  solving. 


Why  Simulation  isn't  Enough 

The  simulation  we've  described  in  the  seven  points  above  could  be  designed  to  simply  minor  the 
reality  of  a  typical  business  environment.  It  would  present  the  cause  and  effect  relationships  found 
in  the  real  world  and  students  would  move  through  its  simulated  environment  and  attempt  to 
discover  these  relationships.  For  example,  by  treating  customers  poorly  several  times  in  the 
simulated  world,  students  are  likely  to  discover  that  this  type  of  action  lowers  revenues. 

Unfortunately,  a  simulation  that  simply  presents  an  analog  of  the  real  world  will  suffer  from  the 
same  limitations  the  real  world  has  in  playing  the  role  of  educator.  As  people  move  through  the 
real  world  they  often  fail  to  learn  important  relationships  (e.g.,  shaking  hands  can  spread  the 
common  cold),  they  often  fail  to  put  their  knowledge  into  practice  (e.g.,  washing  hands  before 
eating),  they  often  learn  things  incorrectly  (e.g.,  they  learn  that  vitamin  C  prevents  colds  when  it 
actually  only  lessens  the  severity  of  colds),  and  sometimes  they  even  "learn"  things  that  aren't  true 
(i.e.,  superstitious  behaviors). 

Because  experience  really  isnt  the  best  teacher,  we  wanted  to  add  mechanisms  to  our  simulations 
to  clarify  and  bolster  student  learning.  We  thus  turned  to  the  notion  of  feedback. 


The  Nature  of  Feedback 

In  its  simplest  form,  feedback  can  be  represented  in  the  following  way; 

Organism's  Action  followed  by— —  Environmental  Event 

Feedback  is  some  environmental  event  that  follows  an  organism's  action  and  can  be  used  by  the 
organism  to  evaluate  the  effects  of  its  action.  For  example,  a  child  who  is  scolded  for  making 
noise  is  receiving  feedback  that  creating  too  much  noise  will  lead  to  scolding  and  is  an  undesirable 
behavior. 

Ever  since  human  beings  evolved  to  the  point  where  they  intentionally  and  systematically  prepare*1 
their  young  for  future  roles  in  life,  parents  and  other  adults  have  utilized  feedback  to  mold  the 
behavior  and  clarify  the  knowledge  of  learner?,  When  three-year  olds  approach  an  open  fire  or 
electric  socket,  parents  provide  feedback  by  yelling  "stop,"  When  one  of  these  same  three-year  olds 
points  to  the  postman  and  says  "Dada,"  mother  responds  by  saying,  "No,  that's  the  postman;  daddy 
is  in  the  kitchen." 

Feedback  doesn't  have  to  be  an  intentional  instructional  event.  As  we  mentioned  earlier,  the  real 
world  can  provide  feedback  itself.  The  child  who  puts  her  hand  in  a  candle's  flame  will  get  feedback 
that  flames  cause  pain. 

Because  of  the  limitations  of  real-world  learning,  educators  and  parents  use  feedback  to  help 
learners  avoid  harm  and  speed  the  learning  process. 
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The  Research  on  Feedback 


Despite  feedback's  long  history,  there  is  stitt  confusion  about  how,  when,  and  in  what  earning 
contexts  to  use  it  (Carter,  1 984;  Cobra,  1 985).  This  is  especially  true  in  regards  to  uses  of 
feedback  in  educational  te  iinology  applications  (Clariana,  Ross  &  Morrison,  1992).  Questions 
regarding  the  lirang  of  feedback       immediate  vs.  delayed;  Kulik  &  Kulik,  1988;  Smith,  1988; 
Sturges*  1*73),  the  control  o*  fexIKack  (e.g.,  student  vs.  external  control;  Sales  &  Williams, 
1988:  feciiand,  Taylor,  :3m*l*s.  itayer  nod  Baker,  1985),  the  effect  of  the  type  of  material  to-be- 
leanttd  (eg.,  high  vs.  low  cognitive  load;  Sales,  1988),  and  the  effects  of  individual  differences 
(t\g,,  tec  m  -rf-contatf,  ability,  sex;  Ckriana  &  Smith,  1989;  Schimmel,  1988)  make  it  difficult  to 
offer  a  clear  prescription  on  the  proper  utilization  of  feedback.  In  fact,  even  the  primary  effect  of 
feedNsck  is  unclear.  Same  theorists  emphasize  ?he  motivational  influence  of  feedback,  while  others 
foes.*  hi  tfoa  informational  aspects  (ses  Salea.  1988). 

£>s ; spite  the  confus  on,  font      ?  *      overall  guidelines  that  have  been  put  forth  based  on  the 
» .search  m  feritac*  '0**r  <Mn;  Odmi,  1985;  Viau  &  Clark,  1987).  First,  feedback  should 
general;.;'  U  bf^ttiaiive  mher  fha;*  &m  livational.  Second,  feedback  for  simple  cognitive  material 
should    ,  viinistt  ^  j*  lma*:'<  fatty.  Feedback  for  higher-level  cognitive  material  should  be 
delayed.  Ybird,  in  most  situation  s  feedback  should  be  just  informative  enough  to  provide 
Knowledge  ci  the  correct  respftas*.  However,  when  students  make  an  incorrect  response  or  the 
material  is  <  *ore  cognitive!  •  ^fewandlng,  they  should  be  given  more  elaborate  explanatory  feedback. 
Fourth,  fa*  more  advance}  »i^kn&,  who  may  get  frustrated  with  too  much  irrelevant  'noise/ 
feedback  should  be  less  waswave  and  more  often  delayed.  Students  of  lesser  achievement  should  be 
given  knowledge  of  correct  response,  perhaps  with  additional  motivational  support;  more  elaborate 
feedback  often  overwhelms  low  ability  students  (Clariana  &  Smith,  1988).  Fifth,  in  some 
situations  it  may  be  helpful  to  give  students  the  option  of  receiving  feedback. 

Unfortunately,  none  of  the  research  that  generated  these  guidelines  used  comprehensive  educational 
simulations  as  the  instructional  intervention.  In  fact,  most  of  the  work  above  uses  rather  simple  or 
short  educational  interventions  that  differ  significantly  from  the  approach  taken  in  simulations. 
Obviously,  simulation  designers  will  want  to  be  cautious  in  utilizing  this  literature.  Nevertheless, 
given  that  simulations  present  complex  situations,  the  research  might  suggest  that  delayed  and 
elaborate  feedback  be  used.  In  the  discussion  below,  this  prescription  is  reinforced,  though  for 
different  reasons  than  simply  the  complexity  of  the  material. 

Note  that  the  dearth  of  empirical  research  on  simulation  design  issues  is  not  limited  to  the  area  of 
feedtr^k  mechanisms.  As  Reigeluth  and  Schwartz  have  noted  from  their  review  of  the  literature, 
ttfe\     'pirically  based  prescriptions  have  been  offered  to  guide  the  design  of  instructional 
simulations*  (Reigeluth  &  Schwartz,  1989;  p.  2).  This  lack  may  be  indicative  of  the  holistic 
nature  of  simulations  and  the  fact  that  simulation  design  (i.e.,  creating  a  mimetic  world)  is  more 
art  than  science. 


Overview  of  the  Project 

In  order  to  understand  the  rationale  and  operations  of  our  simulation  feedback  mechanisms,  an 
overview  of  the  project  is  necessary.  This  is  especially  true  because  we  don't  implement  our 
simulations  as  isolated  instructional  events.  Rather,  they  arc  pari  of  a  larger,  integrated  educational 
program.  Thus,  our  feedback  mechanisms  take  many  forms  and  are  provided  by  many  different 
instructional  components. 

The  Classroom,  Inc.  Project  is  a  collaborative  effort  of  the  New  York  City  Public  Schools, 
Morgan  Stanley  &  Co.,  Inc.,  and  Teachers  College,  Columbia  University'.  The  overarching  goal 


'Funding  for  the  Classroom,  Inc.  project  was  made  available  through  grants  from  Morgan  Stanley 
&  Co.,  Inc.,  The  Morgan  Stanley  Foundation,  The  Mariposa  Foundation,  Louis  Calder 
Foundation,  Bear  Steams  &  Co.,  The  Loews  Foundation,  Andrew  W.  Mellon  Foundation, 
American  Express,  and  Chase  Manhattan  Bank, 
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of  the  project  is  to  prepare  high  school  students  for  entry-level  jobs  in  business  settings,  while 
increasing  their  future  educational  options*  The  project  utilizes  computer-based  business  , 
simulations  as  an  instructional  anchor  for  student  learning  and  as  a  logistical  anchor  for  three  other 
project  components:  mentorships,  summer  internships,  and  curriculum  enhancements  (See 
Appendix  A). 

Two  business  simulations  have  been  developed:  a  service  industry  simulation  (i.e.,  a  hotel)  and  a 
manufacturing  business  (i.e.,  a  clothing  manufacturer).  Each  simulation  has  a  total  running  time 
of  about  10  hours  for  its  computer-based  component  and  is  supported  by  other  instructional 
activities  so  that  the  total  instructional  intervention  for  each  simulation  is  spread  over  an  18- week 
semester,  utilizing  about  60  hours  of  instructional  time  (See  Appendix  B  and  C). 

Each  computer  simulation  utilizes  the  Windows  3.0  environment  and  Asymetrix's  Toolbook 
development  platform  to  combine  graphics  and  text  into  a  user-dominated  exploration  of  the 
simulated  business  environment. 

The  instructional  events  and  activities  augmenting  the  simulation  are  crucial  to  the  success  of  the 
simulation  as  a  learning  tool.  We  now  describe  each  of  these  in  detail: 

1 .  Topic  Introductions 

The  simulation  is  preceded  by  class  time  that  introduces  the  major  topic  areas 
to  be  covered  in  each  simulation  module.  Teachers  can  organize  this  class  time 
in  any  manner  they  think  will  be  most  effective.  Note  that  major  topic  areas 
are  delineated  from  the  list  of  learning  objectives.  To  give  an  example,  Module 
1  might  focus  on  the  Management  and  Marketing  learning  objectives. 

2.  What-if  Analysis 

In  each  session  of  the  simulations,  students  have  to  make  numeric  decisions, 
such  as  setting  the  price  of  a  hotel  room  or  determining  how  much  to  pay  the 
housekeeping  staff.  To  help  the  students  understand  the  issues  involved  in 
making  such  decisions,  a  pre-programmed,  what-if  analysis  spreadsheet  is 
available.  Students  use  this  for  a  whole  class  period  at  the  beginning  of  each 
module. 

3.  Simulation  Play 

In  teams  of  three  or  four,  students  play  the  role  of  the  general  manager  of  the 
simulated  enterprise.  Each  simulation  session  runs  30  minutes,  representing 
six  months  of  simulated  time.  At  the  end  of  each  simulated  year — every  two 
sessions— students  get  a  printout  of  their  financial  reports,  a  status  report,  and 
text-based  feedback  on  each  multiple-choice  decision  they  made. 

4.  Teacher-Guided  Discussions 

After  the  students  play  the  simulation  for  two  contiguous  days,  the  teacher 
leads  a  discussion  of  the  major  decisions  the  students  faced  in  the  focal  topic 
areas  (e.g.,  Management,  Marketing),  * 


'Note  that  the  focal  topic  areas  are  represented  in  about  60  percent  of  the  qualitative  (i.e.,  multiple 
choice)  decisions  the  students  face.  So  for  instance  in  the  first  session  of  Module  1,  students  might 
face  3  decisions  on  Management  issues,  1  decision  that  focuses  on  Marketing,  and  one  decision 
that  involves  communication  skills.  This  allows  the  simulation  to  maintain  its  fidelity  (i.e., 
having  different  types  of  things  happening  at  once)  while  it  enables  the  teachers  to  focus  their 
lesson  plans. 
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5 .  Student-Guided  Analysis  and  Planning 

Students  work  in  their  teams  analyzing  their  results  from  the  printouts,  and 
plan  for  their  future  simulation  sessions. 

6.  Teacher  Summary  and  Student  Activities 

The  teacher  summarizes  the  focal  topic  area  and  provides  an  activity  for  the 
students  to  apply  what  they  have  learned 

7 .  Non-Computer  Simulation  Activity 

Because  neither  the  computer  nor  the  simulation  is  an  appropriate  learning 
tool  for  all  circumstances  and  learning  goals,  we  utilize  non  -computer 
exerc  ses  that  are  part  of  the  simulation  fiction,  but  are  not  part  of  the 
computer-based  portion  of  the  simulation.  For  example,  students  have  to  write 
a  letter  to  the  operating  partners  of  the  hotel  persuading  them  to  authorize 
renovations  to  the  restaurant.  This  activity  take*  several  days  and  is  not 
amenable  to  the  computer-based  simulation  format.  Nevertheless,  the  results 
from  this  activity  are  input  into  the  computer-based  fiction  so  that  students 
will  see  the  effects  of  their  work  in  the  simulated  world.  For  example,  some 
teams  will  receive  better  restaurant  renovations  than  other  teams,  ultimately 
leading  to  increased  revenues  and  higher  profits. 

8.  The  Repeating  Cycle 

After  students  complete  the  non-computer  simulation  activity,  they  again  play 
the  simulation  for  two  days  and  follow  that  up  with  three  days  of  classroom 
discussions  and  activities. 


Unique  Overarching  Goals  for  Simulations 

Simulations  are  qualitatively  different  than  other  instructional  interventions.  Where  other 
interventions  focus  on  small,  often  disparate  elements  to  be  learned,  simulations  aim  to  prepare 
students  to  perform  in  a  particular  environment  or  at  least  understand  some  of  the  important 
relationships  within  a  particular  environment. 

In  the  Classroom,  Inc.  project,  the  goal  is  to  prepare  high  school  students  for  entry-level  jobs  in 
the  corporate  world.  There  is  a  hope  that  students  who  play  the  simulation  will  become  more 
familiar  with  the  language  that  is  used  in  the  corporate  environment,  the  structure  of  a  typical 
business  organization,  and  general  business  operations. 

In  essence,  we  want  students  to  know  how  to  react  to  particular  situations  when  they  enter  the 
workplace.  We  want  the  simulation  to  present  such  situations.  Moreover,  we  want  the  students  to 
understand  the  general  issues  inherent  in  such  situations  so  that  they  can  generalize  to  other  similar 
situations.  We  aim  to  accomplish  this  in  four  ways: 

1 .  Give  students  experience  with  the  language  (and  attitudes)  of  the  corporate 
world 

2.  Give  students  a  feel  for  the  major  structural  fer*ures  within  any  business 
entity. 

3.  Give  students  an  introduction  to  a  large  number  of  general  contingencies 
that  operate  in  the  business  environment. 

4.  Give  students  a  feel  for  how  these  contingencies  are  connected  to  one 
another  and  provide  students  with  an  understanding  for  how  these 
contingencies  are  related  to  tha  important  goals  inherent  to  any  business 
entity. 
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These  learning  mechanisms  are  based  on  the  simple  notion  that  creating  a  simulated  world  will 
help  students  learn  about  that  world*  Of  course,  creating  a  simulated  world  is  not  a  simple  task, 
nor  are  attempts  to  do  so  universally  successful  Nevertheless,  a  large  part  of  the  success  of  a 
simulation  depends  upon  how  well  the  simulated  reality  can  convey  the  important  features  of  the 
real-world  environment. 

Each  of  these  four  areas  will  now  be  described  in  more  detail. 


Giving  the  Students  Experience  with  the  Language  of  Business 

Experience  with  language  and  the  attitudes  inherent  in  a  particular  language  swarai  —in  this  case 
the  language  used  in  the  business  world — does  more  than  simply  lower  the  cojgnfove  workload  of 
students  entering  the  workplace,  In  addition,  language  provides  clues  to  he  a  -r  tingencies  of  the 
environment,  especially  Uc  social  environment  of  the  workplace.  For  example,  lets  take  a  look  at 
the  language  of  the  simui  ited  marketing  manager. 

"Advertising  is  only  worth  so  /nuch.  If  e  don't  treat  out  customers 
properly,  we  are  going  io  lo*:  occupancy  rate,  and  if  our  xcupancy  rate 
falls,  we're  going  to  lose  rgv^  -es.  We've  got  to  implement  a  customer- 
service  training  program  for  our  employees  right  now!  Sure,  it  will  cost 
money,  but  it  will  be  a  gor>u  investment  in  our  future.  * 

This  monologue  is  full  of  direct  lining  points: 

-  Advertising  is  valuable,  but  not  infinitely  so. 

-  Treating  customers  properly  is  essential  to  business  success. 

-  Treating  customers  improperly  lowers  occupancy  rate. 

-  Lowering  the  occupancy  rate  lowers  revenues 

-  Etcetera... 

In  addition  to  the  learning  points  directly  obtainable  from  parsing  the  text,  there  are  additional, 
more  indirect  wisdoms: 

-  TV  mar keting  manager  is  concerned  about  customers. 

-  The  marketing  manager  is  concerned  about  the  business  and  is  willing  to 
make  a  speech  about  it. 

-  Employees  sometimes  need  training  to  perform  at  their  best,  and  this 
training  is  likely  to  benefit  the  business. 

-  It  is  bad  to  lower  the  revenues. 

-  Employees  in  organizations  have  control,  or  at  least  influence,  over  the 
success  of  the  business. 

-  In  the  business  environment,  logical  arguments  are  the  norm  and  are  a 
valuable  tool  in  decision-making. 

Major  Structural  Features  in  the  Business  Environment 

At  the  macro  level,  all  businesses  do  pretty  much  the  same  things  for  the  same  reasons. 
Businesses  are  concerned  with  making  profits.  Profits  are  generated  by  bringing  in  more  money 
than  is  spent.  All  businesses  have  customers.  People  or  organizations  will  only  become 
customers  if  a  business  has  a  product  or  service  that  they  want  or  need.  Because  of  these  inherent 
constraints  on  a  business,  different  businesses  tend  to  develop  similar  structural  features.  To  create 
an  interface  with  customers,  most  companies  have  a  marketing  or  sales  function.  To  keep  track  of 
the  money  situations,  most  companies  have  developed  an  accounting  or  finance  function. 

One  of  the  goals  of  the  project  is  to  familiarize  students  with  typical  business  environments.  Part 
of  this  familiarization  involves  giving  students  an  understanding  of  the  operations  and  functions  of 
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different  organizational  sub-units.  For  instance,  students  should  understand  what  the  marketing 
department  does  and  why  it  does  it 

Alternatively,  these  structural  features  can  actually  provide  students  with  a  reminder  about  some  of 
the  important  goals  of  a  business  organization.  For  example,  by  highlighting  the  fact  that  the 
marketing  department  exists,  students  are  reminded  that  customers  are  crucial  to  business  success. 
In  a  sense,  structural  features  provide  students  with  redundant  cues  about  why  or  how  a  business 
works.  Note  that  due  to  the  static  nature  of  these  structural  features,  their  presence  is  not  sufficient 
to  create  student  learning.  They  must  be  integrated  into  the  interplay  of  the  cause  and  effect 
relationships  of  the  simulated  business. 

To  help  students  learn  about  the  various  functional  areas  in  the  business,  we  allow  the  students  to 
take  meetings  with  their  senior  staff.  These  senior  managers  each  have  responsibility  for  a  major 
functional  area:  marketing,  finance,  engineering,  human  resources,  etc.  Managers  talk  about  a 
particular  issue  from  the  perspective  of  their  functional  responsibility.  For  example,  the  finance 
manager  tends  to  be  concerned  about  cutting  costs;  the  marketing  manager  is  concerned  with  the 
customers;  and  the  human  resource  manager  is  concerned  about  the  employees  of  the  firm. 


Environmental  Contingencies  as  Learning  Objectives 

In  the  Classroom,  Inc.  simulations,  specific  learning  objectives  focus  on  environmental 
contingencies  and  the  causal  effects  of  student  behavior.  For  example,  the  following  learning 
objectives  are  specified  (among  hundreds  of  others): 

-  Students  will  understand  that  successful  companies  understand  and  anticipate 
their  consumer  or  client  needs  and  develop  products  and  services  to  meet 
those  needs. 

•   Students  will  understand  that  managers  can  motivate  their  subordinates  to  be 
productive  by  treating  subordinates  fairly  and  equitably 

-  Students  will  understand  that  having  raw  materials  available  in  inventory 
can  lead  to  smoother  and  more  efficient  scheduling. 

-  Students  will  understand  that  good  work  in  one  job  can  lead  to  a  promotion, 
more  interesting  responsibilities,  and  increased  pay. 

-  In  making  decisions,  students  will  gather  the  appropriate  amount  of 
information,  recognizing  that  good  decisions  require  sufficient  information 
and  that  the  process  of  gathering  information  requires  an  expenditure  of 
time,  effort,  and  sometime  money. 

Note  that  all  of  these  objectives  contains  a  direct  or  implied  environmental  contingency.  In  the 
first  objective  one  contingency  is,  MIf  a  company  meets  the  needs  of  its  clients,  it  will  be 
successful."  In  the  last  objective  one  contingency  is,  "Gathering  an  appropriate  amount  of 
information  will  lead  to  good  decisions." 


The  Network  of  Contingencies  and  Goal-Plan  Hierarchies 

If  teaching  the  contingencies  inherent  in  our  learning  objectives  was  sufficient  to  prepare  students 
for  the  workplace,  we  could  simply  provide  drill  and  practice  instruction  on  the  condition-action 
connection.  For  example,  we  could  teach  that  meeting-the-needs-of-clients  leads  to  increased-sales- 
revenues.  There  are  several  reasons  that  this  is  insufficient. 

First,  learning  condition-action  rules  is  boring.  Students  are  not  motivated  by  such  sterile  learning 
methodologies. 

Second,  the  contingencies  need  to  be  interconnected.  For  example,  in  the  hotel  simulation, 
encouraging  employees  to  treat  customers  properly  leads  to  an  overall  improvement  in  customer 
service,  which  leads  to  an  increase  in  occupancy  rates,  which  leads  to  an  increase  in  revenues, 


801" 


which  leads  to  an  increase  in  profits*  Actions  in  the  real-world  have  multiple  and  often  chained 
effects.  Our  teaching  should  convey  this  fact 

Third,  the  contingencies  are  worthless  if  they  are  isolated  from  meaningful  goals.  Businesses  have 
a  complex  web  of  interconnected  goals,  as  do  the  people  that  work  in  those  businesses.  In  a  sense, 
goals  are  what  drive  the  contingency  engine.  This  is  true  because  actors  (whether  they  are  people  or 
organizational  entities)  only  act  in  the  service  of  meeting  goals.  To  illustrate  these  points,  our 
contingency  involving  meeting-the-needs-of-the-clients  will  only  by  activated  if  other  higher-order 
goals  have  been  activated.  If  students  dont  know  why  they  should  meet  client  needs,  they  wont 
even  try.  We  can  trace  a  series  of  goal-plan  steps  that  would  activate  the  need  for  the  condition  of 
our  contingency  (i.e.,  meeting  client  needs): 

•  Profits  are  a  paramount  goal 

•  Profits  can  be  increased  by  raising  revenues 

•  Revenues  can  be  increased  by  getting  more  clients 

•  Clients  can  be  obtained  by  attempting  to  meet  their  needs 
•  Et  cetera*** 

Students  who  have  internalized  the  higher-order  goals  of  a  business  will  better  understand  why  they 
should  take  certain  lower-order  actions. 

Note  how  the  subgoals  are  not  uniquely  qualified  to  fulfill  the  superordinate  goals.  For  example, 
raising  revenues  is  not  the  only  way  to  get  increased  profits.  Thus,  despite  the  simple  instantiation 
described  above,  an  understanding  of  business  functioning  requires  a  very  complex  goal-plan 
hierarchy,  certainly  one  that  could  not  be  taught  with  any  efficiency.  Simulations  provide  a  vehicle 
for  communicating  this  complex  web  of  understanding  by  enabling  students  to  participate  in  goal- 
directed  actions  and  thinking. 

Finally,  teaching  singular,  isolated  contingencies  is  also  insufficient  because  of  the  inert 
knowledge  problem  (Whitehead,  1929).  As  stated  before,  our  overarching  project  goal  is  to  prepare 
students  for  the  workplace.  In  doing  this  we  don't  want  to  simply  fill  their  heads  with  facts. 
Instead,  we  would  like  them  to  know  what  to  do  when  they  are  faced  with  a  particular  workplace 
situation.  Thus,  our  goal  as  instructional  designers  is  to  strengthen  connections  between  workplace 
situations  and  appropriate  cognitive  activity  and  behavioral  actions.  We  want  to  situate  the 
learning  (Brown,  Collins,  &  Duguid,  1989)  in  the  workplace,  or  at  least  in  a  simulated  workplace 
that  provides  appropriate  hooks  into  the  cognitive  processing  of  the  learner— hooks  that  will  be 
activated  later  when  students  enter  the  real  workplace. 

An  example  may  be  helpful  here.  Certain  business  operations  only  make  sense  when  the  higher- 
order  goals  of  a  business  are  understood.  For  example,  to  a  naive  student,  it  might  not  make  sense 
for  a  company  like  LL  Bean  to  give  a  customer  a  new  shirt  when  a  seam  ripped  in  the  old  one 
after  two  years  of  wear.  On  the  other  hand,  if  the  student  understands  that  customers  provide 
revenues,  and  happy  customers  provide  more  revenues,  the  LL  Bean  policy  makes  sense. 

Any  simulation  that  recreates  the  real-world  environment  with  some  fidelity  will  help  the  learner 
understand  the  major  cause  and  effect  relationships  (i.e.,  contingencies)  of  that  environment.  This, 
of  course,  is  one  of  the  prime  rationales  for  using  simulations  in  the  first  place.  Nevertheless, 
simulation  designers  can  insure  that  these  contingencies  are  understood  by  learners  by 
demonstrating  the  interconnections  between  contingencies  and  goals. 

In  the  Classroom,  Inc.  hotel  simulation,  we  include  an  income  statement  that  highlights  the 
revenue-minus-cost-equals-profits  notion  and  a  balance  sheet  that  highlights  the  asset-vs-liabiiity- 
and-equity  notion.  Moreover,  the  students  learn  that  their  decision-making  in  all  aspects  of 
company  operations  affects  these  financial  statements. 
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Creating  a  Dramatic  Fiction 


We  have  sketched  a  rough  outline  of  some  of  the  theoretical-based  rationales  for  our  simulation 
design.  Now  we  turn  to  a  more  holistically-based  rationale,  but  one  we  think  is  equally  as 
compelling. 

From  a  user-interface  perspective-arguably  the  most  important  perspective  for  instructional 
design-the  defining  characteristic  of  a  simulation  is  that  it  creates  a  fiction!  Not  a  fiction  in  the 
sense  of  untruth,  but  rather  a  fiction  in  the  sense  of  a  coherent,  realistic  interplay  of  characters  and 
events.  Thus,  a  simulation  should  create  a  realistic,  coherent  world  that  plays  out  like  an 
interactive  theatrical  performance  in  which  the  user  plays  the  role  of  the  protagonist.  The 
simulation  should  encourage  users  to  get  lost  in  the  fiction  (see  Laurel,  1986). 

The  importance  of  fictionality  is  twofold.  First,  it  is  motivating  and  interesting.  It  encourages 
users  to  attend  to  the  environment  and  experience  their  situation  holistically  (i.e.,  realistically). 
Second,  the  fiction  provides  a  flow  of  events  that  is  at  least  partially  isomorphic  to  the  world  for 
which  the  students  are  being  prepared.  The  goal  for  the  designer  is  to  insure  that  the  user  gets  lost 
in  the  fiction,  at  least  for  a  while.  Anything  that  takes  away  from  this  first-personness  or  the 
seductive  fidelity  of  this  theatrical  experience  should  be  avoided 


The  Classroom,  Inc.  FetJback  Mechanisms 

We  have  now  reviewed  some  of  the  rationale  we  used  for  designing  the  Classroom,  Inc., 
simulations.  Based  on  these  design  imperatives,  we  wanted  our  feedback  mechanisms  to  do  the 
following: 

1 .  Feedback  mechanisms  should  support  the  user's  intuitions  about 
being  a  part  of  the  simulation  fiction. 

2.  Feedback  mechanisms  should  clarify  misunderstandings  and 
strengthen  previous  learning,  especially  in  regard  to  the 
environmental  contingencies  we  want  students  to  learn. 

3.  Feedback  mechanisms  should  help  the  learner  generalize  to  other 
similar  situations. 

Alessi  &  Trollip  (1985)  delineated  two  forms  of  feedback  in  simulations:  natural  and  artificial. 
Natural  feedback  is  representative  of  real-world  cause  and  effect  relationships  (i.e.,  it  is  the  effect), 
whereas  artificial  feedback  is  provided  as  a  supplement  to  those  realistic,  simulated  relationships. 
This  distinction  is  useful  because  it  describes  how  feedback  can  originate  from  the  simulation 
model  (i.e.,  the  cause  and  effect  relationships)  or  as  augmentation  to  that  model.  However,  as  our 
experience  will  suggest,  a  second  dichotomy  may  be  more  useful  from  a  design  perspective. 

In  keeping  with  our  goal  to  keep  the  user's  head  in  the  simulation  fiction,  we  found  it  useful  to 
make  a  distinction  between  instructional  feedback  events  that  occur  within-the-fiction  and  those 
that  occur  outside-the-fiction.  Note  that  the  distinction  between  "natural  and  artificial n  is  not  the 
same  distinction  as  "within  and  outside,"  Both  natural  and  artificial  feedback  events  can  take  place 
within  and/or  outside  the  fiction. 

To  illustrate  this  point  by  citing  our  simulations,  an  artificial  index  (e.g.,  hotel  elegance)  is  made 
to  feel  a  part  of  the  fiction  and  a  naturalistic  model  with  implicit  feedback  (e.g.,  what-if  analysis) 
is  utilized  outside  of  the  simulation  play. 

To  reiterate,  fictionality  is  vital  because  it  is  motivating  and  provides  students  with  enough 
experiential  fidelity  (i.e.,  the  sense  that  they  are  part  of  a  coherent,  realistic  world)  to  create  transfer 
to  the  real  world  being  simulated.  Feedback  mechanisms  should  be  designed  to  maintain 
fictionality  at  the  same  time  they  augment  the  experiential  learning  process. 
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Utilizing  the  dichotomy  between  within -the-fiction  and  outside-the-fiction,  we  now  turn  to  the 
feedback  mechanisms  used  in  the  Classroom,  Inc.  simulations: 

Within  the  Fiction 

-  Follow-up  Scenarios 

(specific,  realistic  effects  of  multiple  choice  decisions) 

-  Qualitative  Variables 

(indices  of  important  business  concerns) 

-  Financial  Reports 

(cumulative  financial  effects  result  in  changes  on  the 
income  statement  and  balance  sheet) 

-  Variable-based  Scenarios 

(cumulative  financial  effects  activate  scenarios) 

Outside  the  Fiction 

-  Text-based  Feedback  Report 

(specific  feedback  on  multiple  choice  decisions) 

-  What-if  Analysis 

(to  see  the  financial  effects  of  changes  in  numeric 
business  decisions  and  qualitative  variables) 

-  Student  Discussions 

(students  provide  each  other  with  additional  feedback) 

-  Teacher-Guided  Discussions 

(teachers  lead  discussions  on  simulation  scenarios) 

Each  of  these  will  now  be  discussed  in  more  detail. 


Feedback  Within  the  Simulation  Fiction 

As  we  described  above,  the  simulation  fiction  provides  feedback  that  feels  naturalistic,  especially  in 
its  emphasis  on  demonstrating  real-world  contingencies.  In  other  words,  student  actions  have 
realistic  effects.  This  naturalistic  feedback  is  vital  to  simulation  design.  In  fact,  if  you  don't  have 
this  type  of  feedback,  you  donH  really  have  a  simulation. 

Feedback  within-the-fiction  should  feel  naturalistic,  but  it  doesn't  have  to  have  a  precise  real-world 
analog.  For  example,  in  the  Classroom,  Inc.  hotel  simulation,  students  are  given  an  index  that 
measures  worker  morale  on  a  1  to  100  scale.  Worker  morale  is  a  realistic  concept,  but  having  it 
measured  on  a  100  point  scale  is  not  likely  to  be  part  of  a  general  manager's  everyday  experience. 
Thus,  having  a  worker  morale  index  is  artificial,  but  it  works  within  a  coherent  fictional 
framework  because  worker  morale  is  a  concept  that  is  meaningful  to  students  and  consistent  with 
the  real-world  environment 

We  must  add  a  caveat  here  in  regard  to  the  fidelity  of  simulation  feedback.  Although  one  goal  is  to 
provide  naturalistic  feedback,  often  this  feedback  must  be  a  less  transparent  version  of  the 
contingency  in  the  real-world  environment.  For  example,  in  a  real-world  situation  in  which  a 
person  makes  a  social  faux  pas,  he  might  not  learn  the  link  between  his  actions  and  some  delayed 
consequences — he  may  not  understand  why  he  is  overlooked  for  promotion.  In  the  simulation,  the 
link  has  to  be  made  more  explicit.  One  way  to  accomplish  this  is  to  make  the  naturalistic  feedback 
more  potent  or  more  direct.  A  second  way  to  make  this  more  apparent  would  be  to  add  feedback 
outside  the  fiction. 

The  Classroom,  Inc.  simulations  utilize  four  major  forms  of  within-the-fiction  feedback:  follow- 
up  scenarios,  qualitative  variables,  financial  statements,  variable-based  scenarios. 
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Follow-up  Scenarios 


Student  responses  to  multiple-choice  questions  usually  activate  additional 
scenarios.  For  example,  in  the  hotel  simulation,  if  students  fire  their  excellent 
food  &  beverage  manager,  they  will  then  find  themselves  faced  with  the 
deteriorating  performance  of  their  restaurant.  Specifically,  a  scenario  might 
appear  that  has  customers  complaining  about  the  food  and  the  service,  employees 
quitting,  and  restaurant  costs  rising,  This  type  of  feedback  demonstrates  the 
realistic  contingencies  in  the  workplace  in  a  very  powerful  way — students  who 
are  involved  in  the  decision-making  get  to  see  the  results  of  their  own  actions. 

Qualitative  Variables 

Using  an  object-oriented  programming  approach,  a  small  set  of  variables  are 
tracked  during  the  simulation.  These  variables  are  chosen  because  they  are 
meaningful  to  the  industry  being  sii  aulated  and  because  they  are  relevant  to  the 
learning  objectives.  For  example,  in  the  hotel  simulation  customer-service, 
worker-morale,  hotel-elegance,  and  occupancy-rate  are  used  as  qualitative 
variables.  By  playing  the  simulation,  students  learn  that  customer  service  is 
important  in  determining  occupancy  rate  and  occupancy  rate  if  'mportant  in 
determining  revenues.  Moreover,  because  these  variables  are  highlighted  on  the 
status  report,  the  students  come  to  realize  that  these  are  important  goals  to  bear 
in  mind  as  they  work  through  their  decision-making  processes. 

This  feedback  is  available  during  the  simulation  in  the  form  of  the  company's 
status  report,  but  it  is  also  printed  out  for  the  students  later  at  the  end  of  the 
simulated  year  (i.e.,  every  two  sessions).  Students  often  view  this  feedback 
during  play,  without  any  apparent  break  in  the  fiction. 

Financial  Statements 

As  in  most  business  simulations,  students  receive  an  income  statement  and 
balance  sheet  at  the  end  of  each  year  to  help  them  understand  the  bottom-line 
financial  effects  of  their  decisions.  This  type  of  feedback  provides  breadth  by 
acting  as  a  summation  of  student  responses.  It  also  encourages  students  to  stay 
focused  on  some  of  the  primary  goals  in  business;  namely  revenues,  costs,  and 
profits. 

Variable-Based  Scenarios 

As  described  above,  the  qualitative  variables  (e.g.,  customer  service,  occupancy 
rate)  are  tracked  in  real  time,  using  an  object-oriented  programming  approach. 
Scenarios  can  be  activated  based  on  the  level  of  these  variables.  For  example,  if 
worker-morale  drops  below  30,  a  scenario  might  be  activated  in  which  the 
human  resource  manager  sends  the  users  a  report  describing  increased  employee 
turnover.  Employees  might  appear  in  the  lounge,  complaining  about  the 
miserable  state  of  the  company  and  the  poor  management.  Like  the  financial 
statements,  this  mechanism  provides  feedback  based  on  a  trend  of  accumulated 
responses.  Unlike  the  financials,  this  feedback  is  more  specific  to  the  day-to-day 
activities  within  the  simulated  company.  This  type  of  feedback  highlights  the 
instantiation  of  the  qualitative  variables,  showing  students  the  realistic  effects  of 
changes  in  these  factors. 
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Feedback  Outside  of  the  Simulation  Fiction 


As  we  described  above,  the  most  important  feedback  for  a  simulation  is  that  feedback  which  is  a 
natural  part  of  the  simulated  environment.  Nevertheless,  simulations  should  be  augmented  with 
additional  feedback  to  increase  the  effectiveness  and  efficiency  of  the  simulation  in  providing 
redundant  clues  to  the  users.  Similarly,  the  simulation  should  provide  additional  feedback  to  help 
the  students  generalize  their  learnings  to  other  contexts  beyond  the  particular  situation  they  were 
faced  with  in  the  simulation. 

Like  in  the  real-world  environment,  students  moving  through  the  simulation  may  miss  or 
misunderstand  some  of  the  learnings  they  are  given  an  opportunity  to  ir  vrnalize.  To  combat  this 
tendency,  outside-the-fiction,  elaborative  feedback  can  provide  additional  acquisition  opportunities 
and  clarify  misunderstood  concepts, 

The  simulation  should  also  help  the  students  generalize  appropriately.  For  example,  suppose  a 
simulated  employee  yells  at  a  customer,  and  the  student  users  appropriately  fire  that  employee. 
Suppose  that  a  similar  situation  occurs  in  the  future  and  the  students  inappropriately  fire  the 
employee.  The  only  difference  in  these  situations  is  that  in  the  second  case  the  employee  was  brand 
new  and  hadn't  taken  the  customer-service  training  course.  The  key  here  is  that  outside-the-fiction 
feedback  could  be  used  to  help  the  students  understand  the  critical  issues,  discriminating  one 
situation  from  another  based  on  subtle  differences. 

The  simulation  should  also  help  the  students  understand  the  higher-order  goals  they  can  use  to 
generate  reasoned  actions  in  future  unfamiliar  contexts.  Feedback  can  include  general,  goal-based 
principles  that  will  encourage  the  acquisition  of  non-situation-specific  concepts.  For  example, 
suppose  a  simulated  customer  is  quoted  a  room  rate  of  $25  per  night— well  below  the  standard  $85 
per  night.  The  students  decide  to  charge  the  customer  $85  because  they  feel  the  $25  is  clearly  a 
mistake  the  customer  should  have  recognized.  Of  course,  the  customer  gets  angry  and  walks  out. 
By  including  feedback  that  includes  the  general  principal,  Mgo-out-of-your-way-to-please-the- 
customer"  and  and  its  goal-based  rationale,  ttbecause-customers-provide-revenues,w  the  learning  is 
likely  to  transfer  to  a  situation  in  which  a  customer  wants  a  non-perfumed  soap. 

The  Classroom,  Inc.  simulations  utilize  four  major  forms  of  outside-the-fiction  feedback:  text- 
based  feedback,  what-if  analysis,  student  discussions,  teacher-guided  discussions. 

Text-Based  Feedback  to  Multiple  Choice  Questions 

Text-based  feedback  is  provided  for  each  multiple-choice  alternative  the  students 
might  choose.  This  feedback  is  available  on-line  after  a  decision  is  made  and  is 
printed  out  at  the  end  of  each  simulation  session.  This  feedback  not  only 
provides  information  about  the  correctness  of  the  students'  response  in  a 
particular  situation.  It  also  provides  a  general  rule  and  a  contextualized 
understanding  of  the  principle  involved.  In  other  words,  the  feedback  describes 
when,  where,  and  why  the  response  is  good,  bad,  or  mediocre.  In  addition,  this 
feedback  details  the  likely  effects  of  the  decision  on  the  internal  health  of  the 
organization  (i.e.,  as  measured  by  the  qualitative  variables)  and  the  f  nancial 
health  of  the  organization  (i.e.,  as  described  by  revenues,  costs,  an**  profits). 

Although  this  feedback  is  available  during  the  playing  of  the  simulation  (i.e., 
during  the  fictional  drama),  students  are  discouraged  from  looking  at  it  then 
because  of  the  aggressive  demand  on  their  time  and  the  fact  that  they  will  receive 
a  hard  copy  of  it  later  to  perm*  at  their  leisure. 

What-if  Analysis 

The  What-if  Analysis  provides  students  with  a  way  to  get  feedback  about  their 
numeric  business  decisions  (e.g.,  price  of  rooms,  budget  for  advertising,  salary 
for  housekeepers).  By  trying  different  decisions  and  seeing  how  these  decisions 
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might  affect  their  income  statement  and  balance  sheet,  students  learn  about  the 
subtle  relationships  involved  in  quantitative  decision-making.  For  example,  by 
frying  different  pricing  strategies  (e.g.,  same  as  competitor,  higher  than 
competitor)  sCudents  will  learn  how  their  pricing  effects  the  demand  for  their 
service  (or  product).  The  What-lf  Analysis  also  enables  students  to  see  what  will 
happen  if  they  raise  or  lower  their  qualitative  variables.  For  instance,  students 
can  see  what  will  happen  to  their  revenues  if  customer  service  increases,  or  they 
can  see  what  will  happen  to  their  profits  if  occupancy  rates  go  up. 

The  What-if  Analysis  is  available  during  the  simulation  sessions,  but  it  is  rarely 
accessed  by  students  because  it  takes  too  much  time  to  iteratively  attempt 
different  what-if  analyses.  Students  are  given  a  separate  day  to  use  the  What-if 
Analysis  to  help  them  prepare  for  the  numeric  business  decisions  they  face  in  the 
simulation. 

Student  Discussions 

Students  working  in  teams  provide  feedback  to  each  other  both  during  the 
simulation  play  and  afterward  in  team  meetings  where  they  analyze  their  results 
and  plan  for  the  future.  Different  perspectives  are  considered  as  students  are 
encouraged  to  take  everyone's  ideas  into  account.  These  discussions  help  students 
link  their  understanding  of  the  learning  points  to  ideas  they  might  not  have 
otherwise  considered 

During  the  simulations  the  students  argue  and  discuss  the  decisions  that  face 
them  as  they  cooperatively  play  the  role  of  the  general  manager.  Although  this 
is  not  true  to  the  fiction  in  that  multiple  people  are  playing  the  role  of  one 
person,  because  students  are  discussing  situations  that  are  within  the  fiction  the 
flow  of  dramatic  events  remains  intact. 

Additional  student  discussions  occur  after  the  simulation  play  when  teammates 
analyze  their  various  feedback  reports  and  plan  for  future  simulation  sessions. 
These  discussions  occur  after  the  simulation  play  and  thus  do  not  interfere  with 
the  flow  of  the  dramatic  fiction. 

Teacher-Guided  Discussions 

Teachers  discuss  several  of  the  major  qualitative,  multiple-choice  decisions  the 
students  dealt  with  in  the  most  recent  simulation  sessions.  They  use  this 
opportunity  to  clarify  points  or  focus  student  thinking  toward  certain  learning 
objectives. 

These  discussions  occur  after  the  simulation  play  and  thus  do  not  interfere  with 
the  flow  of  the  dramatic  fiction. 


Summary 

Feedback  mechanisms  have  not  been  well  researched  for  instructional  simulations.  Simulation 
designers  must  rely  on  their  experience  or  theoretically-driven  notions  in  providing  feedback  to 
users.  This  paper  discussed  the  rationale  for  the  design  of  the  Classroom,  Inc.  simulations  and  their 
accompanying  feedback  mechanisms.  The  notion  of  fictionality  was  offered  as  a  way  of 
conceptualizing  and  planning  feedback  for  simulations.  Specifically,  it  was  suggested  that  feedback 
mechanisms  should  be  designed  to  balance  the  need  to  maintain  the  user's  intuition  about  being  a 
part  of  the  simulated  world  and  the  need  to  bolster  and  clarify  specific  learning  points. 
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Classroom  Support 


-  Topic  Introductions 

-  WhaHf  Analysis 

-  Teacher-Guided  Discussions 

-  Student-Guided  Analysis/Planning 

-  Student  Activities 

-  Non-Computer  Simulation  Exercises 


Extra-Curricular  Support 


~  Student  Mentorshlps 

-  Summer  Internships 

-  Curriculum  Changes  In  Other  Classes 
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Introduction 

A  recent  development  in  secondary  education  in  the  United  States  has  been  the 
production  and  delivery  of  television  news/information  programs  specifically 
directed  towards  students  between  the  ages  of  12  and  18.  These  shows  are  designed 
to  be  delivered  to  the  students  during  the  school  day.  In  most  cases  programming  is 
made  available  without  cost  to  the  schools.  The  Whittle  Corporation  with  its 
Channel  One  programming  has  been  one  of  the  m^jor  contributors  to  this 
development.  Channel  One  is  unique  in  that  the  the  television  sets,  video  recorders 
and  other  supporting  equipment  are  also  provided  at  no  cost  to  the  schools.  The  costs 
of  producing  and  delivering  the  ten  minutes  of  daily  news  and  information,  the 
equipment  located  in  each  school,  and  the  supporting  programming  and  materials 
are  covered  through  the  sale  of  advertising  spots  embedded  in  the  news  program. 

Since  its  beginning  in  1989-1990,  Channel  One  has  been  controversial  in  the 
educational  community.  A  great  deal  of  debate  concerning  the  impact  such  a 
program  might  have  on  students  and  schools  has  ensued.  Because  it  is  a  new 
approach  in  education,  very  little  research  data  has  been  available  upon  which  to 
base  decisions.  With  little  objective  data  with  which  to  work  the  debates  have  often 
been  marked  by  emotional  stands  on  both  sides  of  the  issue.  Many  national 
organizations  related  to  education  have  publicly  opposed  the  use  of  Channel  One  in 
schools.  Meanwhile,  the  Whittle  Corporation  and  other  supporters  have  pointed  to 
information  largely  gathered  by  the  Whittle  Corporation  itself,  that  shows  the 
programming  is  effective  and  is  well  received  at  the  local  level. 

Research  Questions 

This  paper  describes  the  results  of  a  year  long  study  that  examined  the  impact  of 
Channel  One  on  students  and  teachers  in  a  school  district  in  Minnesota.  The 
students  and  teachers  were  assessed  on  their  knowledge  of  current  events  and 
United  States  geography.  The  results  were  compared  to  those  of  students  in  similar 
schools  in  Minnesota  but  which  had  not  been  receiving  the  Channel  One 
programming.  In  addition,  the  study  attempted  to  assess  student  and  teacher 
reactions  and  attitudes  toward  the  commercials  on  Channel  One. 

The  primary  objectives  of  the  study  were  to: 

1.  Determine  if  a  correlation  exists  between  the  reception  and  non-reception 
of  Channel  One  programming  in  the  schools  on  student's  general 
knowledge  of  national  and  world  current  events. 

2.  Determine  if  a  correlation  exists  between  the  reception  and  non-reception 
of  Channel  One  programming  in  the  schools  on  student's  ability  to  correctly 
position  United  States  locations  on  an  outline  map. 

3.  Assess  the  attitude  of  students  towards  Channel  One  commercials. 


Four  hundred  forty-nine  students  in  grades  6  through  12  from  three  Minnesota 
public  schools  receiving  Channel  One  were  selected  as  the  experimental  group. 
Four  hundred  four  students  in  grades  6  through  12  from  three  Minnesota  public 
schools  not  receiving  Channel  One  were  selected  as  the  control  group.  Care  was 
taken  to  match  the  schools  as  to  size  anc  socioeconomic  base  of  the  community. 
Media  specialists  r.nd  teachers  in  the  respective  schools  selected  intact  classes  of 
students  for  participation  in  the  study. 

In  addition,  fifty-two  teachers  from  the  same  schools  were  included  in  the  study. 
Twenty-seven  of  the  teachers  were  from  the  schools  in  the  experimental  group  and 
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the  remaining  twenty-five  teachers  were  from  the  schools  in  the  control  group.  All 
teachers  volunteered  to  participate  in  the  study. 


Methodology 

Students  and  teachers  in  the  experimental  and  control  groups  were  administered 
identical  written  surveys  to  assess  their  knowledge  of  national  and  world 
information.  Participants  were  asked  to  respond  to  ten  multiple  choice  questions 
concerning  recent  events  in  the  United  States  and  elsewhere  in  the  world. 
Questions  were  written  based  upon  information  available  from  local  newspapers, 
newsmagazines,  and  radio  and  television  news  programs.  Questions  were  written 
only  on  information  or  events  available  from  at  least  two  sources  other  than 
Channel  One.  The  surveys  were  administered  by  the  regular  classroom  teacher. 
Because  of  the  content  of  the  questions  all  surveys  were  administered  within  a  three 
day  time  peviod. 

The  surveys  also  included  an  outline  map  of  the  United  States  and  a  list  of  four 
states  plus  Washington,  D.  C.  Participants  were  instructed  to  mark  the  location  of 
each  state  or  city  on  the  map. 

In  addition,  the  students  and  the  teachers  in  the  experimental  group  were  assessed 
on  their  attitudes  toward  the  commercials  contained  in  the  Channel  One  news 
show.  They  were  asked  to  mark  on  a  five  point  likert-like  scale  their  level  of 
agreement  or  disagreement  with  each  statement. 

Results 

Students  in  the  experimental  group  were  able  to  accurately  position  significantly 
more  U.S.  locations  than  students  in  the  control  group.  There  was  also  a  tendency 
for  the  students  in  the  experimental  group  to  answer  more  questions  correctly  on  the 
test  of  general  knowledge  than  the  students  in  the  control  group,  although  this 
finding  was  not  significant  at  the  .05  level. 


As  expected,  there  was  no  significant  difference  between  the  test  scores  for  the 
teachers  in  the  experimental  group  as  compared  to  the  teachers  in  the  control  group. 
Also  as  expected,  the  teachers  scored  significantly  higher  on  the  general 
knowledge  questions  and  positioning  U.  S.  locations  than  the  students. 


Table  1:  Experimental  vs.  Control  Students 


Experimental  Control 
Mean  Mean 
(n=449)  (n=404) 


tValue  Probability 


General  Knowledge 
U.  S.  Locations 


5.3  4.8  1.95 

3.5  3.0  5.62 


.052 
.000 


Table  2:  Experimental  vs.  Control  Teachers 


Experimental  Control 
Mean  Mean 
(n=27)  (n=25) 


t  Value  Probability 


General  Knowledge 
U.  S,  Locations 


7.8  7,9  .11 

4.6  4.8  ,69 


.914 
,494 
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Table  8:  Experimental  Teachers  vs.  Experimental  Students 

Teachers  Students  t  Value  Probability 

Mean  Mean 
(n=27)  (n=449) 
General  Knowledge  7.8  5.3  2.45  .014 

U.  S.  Locations  4.6  3.5  4.02  .000 

Table  4:  Control  Teachers  vs.  Control  Students 

Teachers  Students   t  Value  Probability 

Mean  Mean 
(n=25)  (n=404) 
General  Knowledge  7.2  4.8  5.86  .000 

U.  S.  Locations  4.8  3.0  6.43  .000 

Statements  concerning  student's  and  teacher's  attitudes  toward  commercials  on 
Channel  are  listed  below  followed  by  the  percent  indicating  the  level  of  agreement 
or  disagreement  with  each  statement. 

The  commercials  on  Channel  1  disrupted  the  classroom  environment 


Agree  1 

2 

3 

4 

5  Disagree 

Students 

10.6 

8.1 

19.6 

17.8 

43.6 

Teachers 

11.1 

3.7 

29.6 

18.5 

33.3 

They  are  the  land  of  commercials  that  keep  running  through  my  mind  after 
Fve  seen  them. 

Agree  1        2        3        4        5  Disagree 
Students  31.2     22.3     22.3      11.1  13.1 

Teachers  7.4     37.0     25.9      18.5  11.1 

The  commercials  reminded  me  that  Fm  dissatisfied  with  what  Fm  using  now 
and  Fm  looking  for  something  better. 

Agree  1        2        3        4        5  Disagree 
Students  6.3     11.1     27.8      22.4  32.4 

Teachers  3.7      3.7      11.1     25,9  55.6 

I  liked  the  commercials  because  they  were  personal  and  intimate. 

Agree  1        2        3        4        5  Disagree 
Students  8.0     10.9     29.5     21.8  29.8 

Teachers  11.1     18.5     29.6       7.4  33.3 

Fve  seen  these  commercials  so  many  times-Fin  tired  of  them. 

Agree  1        2        3        4        5  Disagree 
Students  31.5     19.1     27.4       9.7  12.4 

Teachers  37.0    29.6     18.5      14.8  0.0 

The  commercials  irritated  me-they  were  annoying* 


Agree  1 

2 

3 

4 

5  Disagree 

Students 

13.3 

9.7 

23.0 

23.5 

30.5 

Teachers 

11.1 

11.1 

22.2 

29.6 

25.9 

The  products  advertised  on  Channel  1  are  approved  by  the  teachers. 


Agree  1 

2 

3 

4 

5  Disagree 

Students 

15.7 

17.1 

40.8 

11.8 

14.7 

Teachers 

7.4 

11.1 

29.6 

11.1 

40.7 
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Conclusions 

The  fact  that  students  exposed  to  Channel  One  were  better  able  to  position  U.  S. 
locations  on  a  map  than  students  not  exposed  to  Channel  One  indicates  that  viewing 
Channel  One  is  positively  correlated  with  this  geography  skill  A  possible 
explanation  for  this  is  the  extensive  use  of  maps  and  other  graphics  showing  U.  S. 
locations  on  the  Channel  One  programs.  The  exposure  to  news  and  current  events 
via  Channel  One  could  also  explain  the  student's  tendency  to  answer  more 
questions  correctly  on  current  events  although  this  finding  was  not  significant  at 
the  .05  level  as  indicted  above.  Additional  study  in  this  area  is  need  to  fully  assess 
the  impact  Channel  One  is  having  on  students. 

A  majority  of  the  students  and  of  the  teachers  did  not  find  the  commercials 
disrupting  to  the  classroom  environment  (61%  and  52%  respectively)  or  that  the 
commercials  were  annoying  (54%  and  55%  respectively).    Although  a  majority  of 
both  groups  indicated  they  were  tired  of  the  commercials  (51%  and  67%).  These 
findings  could  be  explained  by  amount  of  exposure  students  and  teachers  have  to 
commercials  in  their  daily  life.  People  may  be  so  accustomed  to  them  they  do  not 
find  them  disruptive  or  annoying  even  though  they  do  feel  tired  of  them.  In  this 
respect,  this  study  seems  to  indicate  from  Channel  One  has  had  very  little  impact 
on  students  or  teachers.  The  finding  that  a  much  greater  percentage  of  teachers 
indicated  a  level  of  "tiredness"  with  commercials  could  be  explained  by  the  fact  that 
the  commercials  are  directed  at  the  age  level  of  the  students  rather  than  adults. 

A  majority  (54%)  of  students  indicated  the  commercials  were  the  kind  that  kept 
running  through  their  mind  tends  to  indicate  the  commercials  are  effective  in 
getting  the  product  into  the  student's  thoughts.  However,  55%  of  the  students 
indicated  that  the  commercials  did  not  remind  them  they  were  looking  for  a  better 
type  of  product.  Fifty-one  percent  of  the  students  also  indicated  they  did  not  think 
the  commercials  were  personal  and  intimate.  This  seems  to  indicate  that 
consciously  students  were  not  looking  at  the  commercials  to  change  their  buying 
habits. 

When  asked  if  they  thought  the  products  advertised  on  Channel  One  were  approved 
by  the  teachers,  the  students  gave  no  clear  indication.  The  largest  group  (41%)  was 
undecided  on  this  issue  with  the  remainder  of  the  students  being  relatively  evenly 
distributed  in  the  other  four  categories.  This  may  be  one  area  that  teachers  will 
want  to  address  in  assisting  students  to  become  mor.  informed  consumers. 
Teachers  may  wish  to  indicated  to  students  that  even  though  a  prou.. , ,  is  advertised 
in  materials  that  are  part  of  the  curriculum,  there  does  not  necessarily  mean  the 
teachers  have  approved  that  product. 

This  study  indicates  that  Channel  One  programming  has  had  a  small,  but  positive 
impact  on  students  in  a  least  one  area  of  geography  skills.  At  the  sp  ue  time,  the 
commercials  seem  to  be  tolerated  as  a  part  of  the  packaging  similar  to  the  way 
commercials  are  viewed  on  commercial  television.  This  might  indicate  a 
blurring  of  the  lines  in  student's  minds  between  formal  education  and  other  aspects 
of  their  lives. 
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A  SURVEY  OF  STUDENT  TEACHERS'  LIBRARY  MEiJJ/V  SKILLS:  A 

REPLICATION 

Literacy  is  defined  as  the  ability  to  read  and  write.  An  extension  of  this 
definition  for  the  information  age  is  "the  abilif  v  to  lo<  it*,  retrieve,  selxt,  organize, 
evaluate  and  communicate  information.  Cast  in  a  slight1  /  Hiff*  *ert  fashion, .  .  .  the 
ability  to  w  ess,  assimilate,  and  transfer  information"  (Thurmon,  19''7).  It  is 
assumed  that  the  hallmark  of  an  educated  person  is  the  ability  to  access 
information  ot  any  topic,  in  a  field  of  interest  or  profession,  assimilate  the  material, 
and  communicate;  i  h<  resulting  information.  This  assumption  is  made,  not.  only  for 
all  students  who  achieve  a  baccalaureate,  but  particula=    f  r  those  who  propose  to 
enter  :he  fieM  of  teaching.  It  will  be  their  task  to  maintain  and  improv   the  literacy 
of  our  children  . 

The  ability  to  access  information  by  studei.;  ♦aacUe»-s  is  the  basis  for  this 
study.  Fu»  if  t^achen:  a;   to  teach  students  to  access  ,  ft  auction,  they  must 
possess  the  bibliographic/mediagraphie  skills  first.  Alti  ugh  this  study  is 
addr  •  wsintf  an  issue  of  the  information  ago,  it  emerged  in  t  n*  literatut*  nearly  thirtv 
years  ago. 

In  196\  when  school  libn  tries  we?^  beginning  the  tr-i  ^formation  nto  lib  ivy 
v\tttVw  enters,  Witt  made  the  folio  ving  observations  about  the  proficiency  U 
U  'ichors'  library  skills: 

"No  one  is  more  influential  than  the  teacher  in  shaping  the 
curriculum  nnd  in  determining  Uvi  nature  and  extent  of  a  learners 
achievements.  Consequently,  if  we  are  to  have  U.urher  who  belie  v< 
it  important  to  use;  many  different  type1-  of  materials  for  teaching  and 
learnn and  who  K  »ow  bov  t».  use  these  materials  apprpri; ly 
and  effectively,  steps,  must  be  taken  to  make  c*'jin\n  that  teach n 
develop  the  values,  knowledges  and  skills  es(  rntml  to  this  *ort  oi 
professional  behavior."  (p,  f>7) 
bu  aig  the  mid-sixties  a  five-year  collaboration  between  selected  teacher  education 
in  titutions  and  public  schools  was  fundeu  t     he  Knaj    Foundation  tn  prov'ie  the 
training  described  by  Witt  (Suilivjv\  1967,  pp  J  '  !5).  A  result  r/ihe  project  was 
(he  construction  of  na  13  item  que^t  toiinair  to  t<  1 1  U  .  'hers'  limwiedge  ot  the 
availability  and  use  of  library  resources  <md  wrvi;*  s"  (Walker,  i*;67,  p  17)  "Eleven 
of  the  13  items  on  the  question*  uvi^e  mm.  foci'  *1  it<  ms.  The  \.  her  two  a:  k  H  the 
stu  dent  teachers  to  name  ways  ihe  lil  -iry  and  le  •  a  nans  could  h<  -/  service  to  them 
ia  their  teaching"  (Walker  19G7,  p.  21-t,.  TH*  qut  -V-mnH*  e  was  given  '  *.  %~  stud*  n* 
t  ichers.  The  resultant  o  La  suggested  two  ir»  nds    1    that  sti  ien<  teach*     use  a 
small  number  of  refi  rences  or  s<  un  is  of  informal  ion;  and  2  the;  1  avt  a  merger 
concept  of  how  the  library  and  the  libi  wria;i  car  help  tr      in  th*ir  Uineni  (Walker 
IMG,  p.  66). 

In  the  fin; 1  ref,  >rt  of  ti    i  -  iect,,  the     *or  slat'  J  tba  there*  were  sev<  id 
iinpiii :  i  ions  for  teach*  r  ulucati'm:      'hat  unit'.*?  f  >»  htw  untier: m<  <j  f  e  library 
•  nction  and  utilize  it1     n^urces,  th  j;  •»  iki  n    \>    vide  in   nation  tor    i<j)  h>ld  o» 
hut  own    v:  I  of'lcai  rinj.;;  in  d  cultural      ^rounu,  i-    2)  .14.  -be  let»  her  '  rmujug 
ir.-;Hf.:3tioi:  ha:  a  responsibility  fc.    -epare  t<  <i.  <   rs  who  u  dfe^  #*nd  4his  role  f  the 
library  and  who  will  m;=ke  use. «  ""all      resource     *  i)Ii\ar>,  \   ,r  p  j/> 

i"  Cttlfe&V  !4UUllkS}^r-htd!SU.   i-uaOL,  au  Amm  ^ftii  i  ibr,  /  v  '  iHOCiation 
f  A«f;A)  eommitt*  ^  >         ri  sim.  «»»  s-s ■•u^.-an*  ti*  ^  'Siut^enu  »v*  ^ducaM s*i  need  an 
increasingly  Hopb^tu'ateii  Unow)e*!g'e mf  -  ijiary  n  <  imss       on    oau^  nn 
mformatioki  in  the  ■■'*htt,,-»lini.  'Wr^i,  c  Iiv>     1  ue  biWi  :r?apis.i-         .tfte  ei^s  for 
i'l^uoatir-ii  Sturi«'n^i    .v..iiifu*e  v,    ^iofred      \  <ruua  ii»  to  r'v<'t»»'tr;  . ■  ■•a^*  »neies 
primarily  hi  ihe  p:OfeK:.e  ■  »1  iutf/i '.f  teae:-*  »    e.r  ,;ivk.n  ......  ;wnt4n  Is 

aecosN- :-fJ  by  tcwltui  -ni  tvajij,.^;  ^ai-he*      «   *::at:i:^fi:  in    rvv:<;<\         fuciUh  *;l  ti^ 
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thirteen  item  instrument  was  the  skills  needed  as  an  undergraduate  or  a  graduate 
student  in  education,  rather  than  the  knowledge  base  needed  by  a  practicing 
teacher,  There  has  been  no  reported  use  of  this  ALA  instrument  in  the  literature. 

Six  years  later  a  tangential  study  in  Ohio  surveyed  "elementary-education 
faculty  at  teacher-training  institutions  in  Ohio     ...  to  explore  attitudes  toward  the 
elementary  school  t;  icher's  role  in  research  skills  instruction  and  toward  library 
instruction  in  teacher-training  programs  .  . .  ,  [since]  elementary  schoolteachers  play 
an  important  part  in  the  develo^inrnt  of  both  bibliographic  skills  and  the 
critical-thinking  skills  that  are  necefc.ar  to  complete  research  tasks  successfully" 
(O'Hanlon,  11*87,  p.  17).  The  survey  resulu  supported  the  teaching  of  library  skills 
to  future  teachers. 

A  search  of  Tesi ,  in  Print  III  to  locate  r     *urement  instruments  for 
determining  library  media  skills  of  student  t         „  yielded  relatively  dHmal 
results.  Three  teals  were  located,  one  for  grade,  /  through  college,  one  v  high  school 
through  college  'Test  on  Use  of  the  Dictionary,"  and  a  set  of  three  library  tests  for 
college.  The  information  for  each  indicated  either  they  were  old,  1980,  1955-63, 
1967-72  respeciively;  or  the  in-print  status  was  uncertain  (Mitchell,  1983,  pp.  214, 
425). 

In  addition  to  the  scattered  and  negligible  research  reported  in  the  literaluie, 
another  force  is  impending  which  will  call  for  the  kind  of  research  study  reported 
here.  Increasingly  states  are  implementing  curriculum  guides  which  suggest 
teachers  teach  study,  reference  or  research  skills,  in  essence  library  media  skills.  A* 
an  example,  The  Arkansas  Curriculum  Guides  include  the  following  basic  skills. 
Location  of  information  using  the  card  catalog,  encyclopedia,  an  index,  and  a  table  of 
contents  (Language  ArU  -Grade  6,  p.  10);  identification  of  the  four  types  of  cards  in 
the  card  catalog  and  location  of  book*;  by  using  call  umbers  (English  L-Grade  9,  p. 
7);  and  the  location  of  information  in  a  bibliography,  using  footnotes,  using  a 
thesaurus,  using  the  Reader's  Guide  and  Who's  Who"  (English  II--Grade  11,  p.  16), 
To  date  there  lias  been  no  measure  of  accountability  included  in  either  teacher 
education  nor  teacher  practice  of  the  skills. 

Therefore,  rather  than  develop  an  instrument  to  measure  student  enrhers1 
library  media  skills,  it  seemed  prudent  to  replicate  the  questionnaire  used  in  tin 
Knapp  Foundation's  Project.  This  was  used  to  evalur'e  me  skills  student  teachers 
took  into  the  field.  The  resi  Its  are  compared  with  those  of  Lhe  original  study  to 
determine  whether  the  findings  remain  stable  with  the  passage  of  time,  or  not.  /  !so 
the  results  are  somewhat  indicative  of  the  staie  of  sludont  teachers'  library  media 
skills,  as  well  as  the  instruction  received  in  institutions  ol  higher  education. 

Thus  the  primary  objective  of  th  *  study  was  to  replicate  lUo  survey  of  tin 
Knapp  Foundation  Project,  edited  and  appended  to  for  currency,  to  a&fady  the  current 
status  of  student  teachers'  library  m<  lia  skills.  A  second, try  objective  was  to 
expand  the  population  studied  to  include  all  teacher  education  institutions  in  the 
state  of  Arkansas,  rather  than  85  subjects  at  experimental  sites  within  three  .states 
as  was  done  initially.  A  tertiary  objective  was  to  aecum  >lnte  a  database  of 
responses  for  subsequent  studies  which  will  include  the  adinirn -tration  of  this 
questionnaire,  in  closed  for. a,  to  a  selected  population  that  is  geographically 
dispersed. 

Methodology 

As  stated  previously,  the  primary  n1  iective  of  this  study  was  to  replhute  the 
survey  of  the  Knapp  Foundation  Project,  to  ussa>  u  <?  "urrent  status  of  student 
Leat  hers'  'ibrury  madia  skills.  Data  for  this  study  wut:*  t.  /II  oted  by  mailing  sets  of 
questii .mi; tires  U>  onuict  people  in  id*  atified  teacher  education  sites,  administered 
i  )    lid*- -its;  just  pr  ior  to  their  field  experience  ami  returned  to  the  University  e< 
Central  Arkansas  for  collation  and  analysis. 
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Subjects 

The  subjects  were  student  teachers  who  began  their  field-experience  in  the 
spring  of  1991  in  Arkansas.  Of  the  19  institutions  listed,  5  of  the  major  schools  of 
education  participated,  with  444  students  responding  to  the  survey. 

Materials 

The  survey  instrument  was  a  22-item,  open-ended  questionnaire,  The  items 
had  been  administered  to  several  graduate  educational  media/library  science 
research  classes  over  a  2-year  span  of  time  to  clarify  the  questions.  However,  the  13 
items  of  the  original  questionnaire  retained  their  initial  intent,  if  not  form,  while 
other  items  were  added  for  administrative  purposes  (3)  or  provision  for  technological 
changes  in  information  acquisition  (6). 

Procedures 

Based  upon  state  department  of  education  lists,  each  school  of  education 
dean  within  the  state  was  contacted  by  letter  to  gain  their  support,  arrange  the 
administration  of  the  questionnaire,  and  identify  a  contact  person;  included  was  a 
postcard  to  return  the  name  of  the  contact  person,  the  time  for  administration  of  the 
survey,  and  number  of  students  that  would  be  participating.  According  to  the 
schedule  of  administration,  sets  of  questionnaires  were  mailed  to  each  contact 
person,  with  a  letter  of  transmittal  and  a  pre-addressed  stamped  envelope  for 
return.  For  any  packages  of  questionnaires  which  were  two  weeks  late,  according  to 
the  schedule  of  return  deadlines,  the  contact  persons  were  telephoned  as  a  reminder. 

Analysis 

The  responses  to  each  question  were  entered  and  frequency  counts  were 
tallied  for  each  institutional  group.  Then  the  resulting  top  10  categories  for  each 
question  from  each  institution  were  merged  into  a  final  frequency  count  tally.  A 
conscious  effort  was  made  to  enter  open-ended  responses  with  a  minimum  of 
interpretation,  to  obtain  categories  for  the  tally.  From  the  pooled  tally,  the  top 
categories  for  each  question  were  ranked.  All  of  the  data  was  pooled,  therefore  no 
linking  of  respondents  and  responses  was  attempted,  i.e.,  according  to  elementary 
education,  secondary  education,  and  fields  of  specialization  respondents,  as  was 
done  in  the  original  Knapp  Foundation  study. 

Results 

To  report  the  results,  the  questions  ami  responses  have  been  assembled  into 
five  content  categories:  Background  information,  Bibliographic  skills,  Mediagraphic 
skills,  Bibliographic/electronic  sources,  and  Perceptions  of  library  and  librarian. 

Background 

The  first  three  questions  were  pot  ed  for  background  and  administrative 
purposes  to  determine  who  the  subjects  were,  what  skills  they  had  learned,  and 
where  they  had  learned  their  skills.  Based  on  the  first  question,  the  major  field  of 
study  in  education,  the  following  response  pattern  emerged:  Elementary  Education, 
245;  Secondary  Education,  184;  and  Other/blank,  15. 

The  next  question  asked  whether  the  student's  college  work  included 
instructions  in  library  skills  or  media  skills  and  in  what  class?  Of  those  responding, 
65%  (272,  n  s  416)  reported  library  instruction,  and  90%  (368,  n  s  411)  media 
i  ;truction.  Instruction  wa  ;  most  frequently  encountered  in  a  special  media  class; 
th* :i«  in  order,  part  of  another  class,  orientation  sessions,  a  special  library  class,  and 
other  classes. 

The  last  question  dealt  with  student  teachers'  perception  of  whether  they 
hiH  'earned  to  use  the  library  (library  media  center)  effectively  in  their  teaching  or 
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not.  Two  hundred  seven  or  48%  (n  =  428)  felt  they  had  learned  to  use  the  library 
effectively  in  methods  classes,  other  classes,  and  special  library  classes. 

Bibliographic  skills 

In  the  next  section  of  the  questionnaire  bibliographic  skills  were  addressed. 
For  example,  one  question  asked  "You  can  locate  a  book  through  the  card  catalog  by 
looking  up  its"  what?  Students  listed  author  (430),  title  (421),  subject  (378),  and 
other  (49).  Indicative  of  the  pattern  of  responses  for  library  skills,  was  the  question 
which  stated:  "If  a  student  wanted  to  see  if  any  magazine  articles  had  been  written 
on  a  topic  you  have  given  him/her  to  explore,  what  reference  works  would  you 
suggest?"  The  top  10  responses,  in  rank  order,  were  the  following;  1.  The  Reader's 
Guide  to  Periodical  Literaturor  2.  EfilQ,  3.  periodicals,  4.  the  card  catalog,  5. 
microfiche,  6.  an  index  of  journals,  7.  a  periodical  catalog,  8.  a  computer,  9.  ask  a 
librarian,  and  10.  microfilm. 

Mediagranhic  skills 

Four  questions  specifically  addressed  media  skills.  First,  students  were 
asked  what  was  "the  mr  in  difference  between  an  opaque  projector  and  an  overhead 
projector."  Of  the  326  responding,  61%  (199)  were  correct,  and  39%  (127)  were 
incorrect.  Then,  the  question  "to  find  out  what  filmstrips  are  available  on  a  given 
topic,  what  reference  would  you  consult?"  was  asked.  The  top  responses,  in  rank 
order,  were:  1.  the  card  catalog,  2.  ask  a  librarian,  3.  an  educational  cooperative 
agency,  4.  the  state  PBS  affiliate,  5.  a  filmstrip  catalog,  6.  a  media  guide,  7.  a  book 
of  filmstrip  titles,  8.  a  library,  and  9.  a  curriculum  lab.  The  third  question  dealt 
with  listing  "constraints  imposed  on  teachers  by  the  1976  copyright  laws  (or 
subsequent  revisions  of  the  law)."  The  responses,  in  rank  order,  were:  1.  get 
permission  to  copy,  2.  the  number  of  copies  that  can  be  made  are  limited,  3.  the  law 
varies  with  the  type  of  material  copied,  4.  copied  materials  must  be  used  in 
classrooms,  5.  no  profit  should  be  made  with  copies,  6.  there  is  a  time  limit,  7. 
materials  cannot  be  copied,  8.  purchase-don't  copy,  and  9,  plagerism.  The  fourth, 
and  last  media  question  was  one  that  was  added  to  reflect  newer  technology:  "what 
source(s)  would  you  use  to  find  appropriate  microcomputer  disks  to  support 
classroom  teaching?"  In  rank  order,  the  top  responses  were:  1.  catalogs,  2.  software 
guides/catalogs,  3.  the  library,  4.  ask  the  librarian,  5.  ERICf  6.  "Only  the  Best" 
magazine,  7.  the  card  catalog,  and  8.  books. 

Bibliographic/electronic  sources 

Six  more  questions  addressed  bibliographic  skills,  two  of  which  crossed  the 
boundary  of  media,  those  of  electronic  reference  sources  and  electronic  databases, 
The  answers  to  one  is  indicative  of  the  other,  when  asked  to  name  an  electronic 
reference  source  which  includes  full  text,  or  name  electronic  databases  that  serve  as 
indexes  to  information,  the  resultant  distribution  was  slightly  different  but  the 
categories  were  the  same.  The  query  on  electronic  databases  gained  the  following,  in 
rank  order:  1.  £E1C/CD-R0M,  2.  computer,  3.  microfiche,  4.  video,  5.  microfilm,  and 
6.  Psych,  lit. 

Perception  of  library  «nH  librarian 

The  last  two  questions,  from  the  original  questionnaire,  reflect  students' 
perception  of  the  library  [library  media  center]  and  the  librarian  [library  media 
specialist].  In  response  to  the  question:  "How  can  the  library  be  of  service  to  you  in 
yoi  r  teaching?",  twenty  catagories  emerged.  The  top  10  catagories,  in  rank  order 
were:  1.  provide/access  media,  2.  have  research  materials  for  students  and  teachers, 
3.  have  references,  4.  have  books/trade  books,  5.  children/teachers  can  get  books,  6. 
have  topic/content  background  resources,  7.  provide  information,  8.  have  ideas  [for 
bulletin  boards,  activities,  etc.],  9.  have  media  equipment,  and  10.  have  many 
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resources.  To  elicit  the  students'  understanding  of  the  librarian,  the  following 
question  was  posed:  "As  a  ^assroom  teacher,  what  competencies  would  you  expect 
the  school  librarian  to  posf    i  that  would  enhance  and  supplement  your  teaching?" 
The  top  replies,  in  rank  order,  were:  1.  know  library  and  use  of  its  materials,  2. 
have  knowledge  of  sources,  3.  be  able  to  access  information,  4.  have  media 
experience,  5.  be  able  to  help  teachers  find  information,  6.  work  with  and  have  an 
interest  in  students,  7.  know  what's  available,  8.  have  a  willingness  to  he1  ) 
students,  and  9.  be  able  to  access  information  for  teachers. 

Discussion 

The  objectives  of  this  study  were  met,  with  varying  degrees  of  success.  The 
Knapp  Foundation  Project  survey  was  edited  and  appended  to  for  currency,  and 
replicated.  The  objective  to  expand  the  population  studied,  had  mixed  results.  The 
number  of  subjects  was  expanded  to  include  444,  by  far  more  than  the  original  85; 
but  the  expansion  to  the  19  teacher  education  institutions  in  the  state  of  Arkansas, 
fell  short  with  5  participating  institutions.  Even  though  the  original  intent  of  this 
study  was  to  survey  the  state  population,  the  dominant  teacher  education 
institutions  did  participate.  The  third  objective,  to  accumulate  response  data  for 
subsequent  studies  was  partially  met;  most  of  the  response  categories  were  too 
general  to  be  used  to  create  a  closed-form  questionnaire,  but  can  be  used  to  guide 
the  construction  of  another  survey  form. 

In  response  to  bibliographic  skill  questions,  the  trend  seemed  to  be  from  a 
limited  set  of  specific  responses  students  probably  learned  prior  to  college,  to  very 
generalized  responses;  overall  students  have  a  good  general  sense  of  information 
genre,  without  knowing  a  range  of  specific  titles.  This  trend  continued  wher 
electronic  bibliographic  resources  were  examined.  Very  likely  the  recent  incorporation 
of  computerized  indexes  and  online  database  technology  has  contributed  to  the 
confusion  of  what  an  electronic  resource  is,  a  computer,  the  CD-ROM  disk,  or  the 
specific  program  driving  the  information  search. 

Responses  to  mediagraphic  skill  questions  varied  greatly.  The  most 
surprising  result  was  the  response  to  the  opaque/overhead  projector  distinction.  The 
correct  replies  either  centered  on  the  type  of  materials  each  projector  used,  the 
lighting  in  the  classroom  when  vsed,  or  the  relative  heat  produced  in  operat  ing  each 
projector.  Then  when  queried  about  filmstrips,  students  seemed  to  be  aware 
generally  of  where  to  find  informatior  but  tended  to  mix  media;  this  probably 
reflects  the  current  emphasis  on  "high  technology"  in  teacher  education  to  the 
detriment  of  other  media.  The  previous  speculation  on  "high  technology"  is  borne  out 
with  the  results  of  the  copyright  question,  since  it  is  often  related  to  copying 
computer  programs  and  videotape  when  addressed  in  methods  coursework. 

The  students'  perception  of  the  library  and  librarian  seemed  to  be  mixed, 
with  the  egocentric  expectation  that  the  library  would  have  all  the  materials  a 
teacher  would  ever  need  to  teach  a  class;  and  then  have  a  perceptive  insight  into  the 
competencies  a  librarian  should  ha^  e.  This  dichotomy  is  probably  due,  in  part,  to  the 
questions  being  framed  in  terms  of  library  and  librarian,  rather  than  library  media 
center  and  library  media  specialist.  There  may  also  have  been  confus(  \ 
expectations  based  upon  students'  experience  in  a  college  library,  and  the  distant 
memory  of  earlier  schooling. 

In  comparison  with  the  two  trends  suggested  by  the  Knapp  study,  today's 
students  still  seem  to:  1.  use  a  small  number  of  references  or  sources  of  information, 
and  2.  have  a  merger  concept  of  how  the  library  and  librarian  can  help  them  in  their 
teaching;  though  their  concept  of  a  "librarian"  is  limited,  it  seems  perceptive. 

In  conclusion,  this  study  was  phase  I  of  a  three  phase  research  project.  In 
phase  II  the  questionnaire  will  be  reformatted  so  that  subjects  can  check  answers,  it 
will  include  media  items  that  deal  with  telecommunications  and  microcomputers, 
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and  the  sample  for  study  will  be  drawn  from  four  states.  In  phase  III  the 
closed-form  questionnaire  will  be  conducted  nationwide. 
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Abstract 

Most  theories  of  instruction  specify  that  students  should  engage  in 
meaningful  interaction  within  an  actual  learning  situation  in  order  to  construct 
their  own  understanding  of  a  domain  of  information.  When  students  do  so,  the 
information  they  glean  will  be  salient,  generalizable  and  less  subject  to  forgetting. 
Learning  from  actual  situations,  however,  is  often  not  practical  and/or  is 
sometimes  expensive  (in  terms  of  costs  versus  benefits)  for  many  learning  tasks. 
Therefore  efficient  learning  sometimes  requires  the  availability  and  use  of 
instructional  simulations  and/or  other  interactive  learning  environments.  Such 
instructional  simulations  are  now  possible  and  readily  available,  thanks  to  the 
advent  of  the  microcomputer. 

In  this  presentation  we  target  four  major  topics  from  psychology  that  are 
relevant  to  the  design  of  microcomputer-based  instructional  simulations.  These 
four  issues  are:  (a)  cognitive  structure;  (b)  cognitive  strategies  &  metacognition; 
(c)  automaticity;  and  (d)  affect.  In  addition  we  put  forth  several  guidelines  for 
creating  instructional  simulations  with  these  topics  in  mind. 
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Building  Microcomputer-Based  Instructional  Simulations: 
Psychological  Implications  and  Practical  Guidelines 

Introduction 

The  advent  of  the  microcomputer  has  resulted  in  an  increased  interest  in 
the  design  and  use  of  instructional  simulations.  This  type  of  simulation  has  the 
potential  to  enhance  the  transfer  of  learning  by  teaching  complex  mental  and 
procedural  tasks  in  an  environment  that  approximates  fairly  realistic  settings 
(Reigeluth  and  Schwartz,  1989).  However  many  microcomputer-based 
instructional  simulations  produced  toda>  cue  poorly  designed  from  a 
psychological  standpoint  (Hill,  1989),  as  d  therefore  do  not  provide  an  effective 
learning  environment. 

There  seems  to  be  a  dearth  ox'  nformation  concerning  how  to  specify  and 
implement  appropriate  instructional  designs  for  this  typn  of  learning  context 
(Alessi,  1988).  This  presentation  represents  our  attempt  at  identifying  areas  in 
the  psychological  literature  which  have  important  implications  for  the 
instructional  design  of  microcomputer  based  simulations.       Following  is  a 
representative  summary  of  the  information  we  have  gleaned  from  research  in  the 
areas  of  training,  instruction,  cognitive  psychology,  and  human  factors.  In  our 
presentation  we  will  present  these  themes  and  concepts  in  a  much  finer  level  of 
detail. 

Cognitive  Structure 

Our  view  of  the  world,  of  ourselves,  of  our  capabilities,  and  of  the  tasks  we 
are  asked  to  perform,  depend  quite  a  bit  on  the  mental  conceptualizations  we 
bring  to  a  task.  As  we  act  and  react  to  the  environmont,  to  others,  and  to  the 
machines  (including  training  devices)  around  us,  we  form  and/or  modify  our 
internal  mental  representations  of  the  things  with  which  we  are  interacting. 
These  statements  hardly  need  to  be  said  but  we  do  so  because  they  represent  an 
underlying  theme  which  will  surface  again  and  again  as  we  discuss  the 
psychology  of  instructional  simulations. 

Implications.  One  of  the  key  questions  a  designer  of  instructional 
simulations  needs  to  ask  is;  "What  constitutes  good  strategies  for  making 
instructional  simulations  conducive  to  improving  cognitive  structure?"  We  give 
several  suggestions  below. 

•  Simulations  should  use  appropriate  imagery,  audio,  vocabulary  and 
organization  for  the  level  of  understanding  the  student  is  at. 

•  Simulations  should  correspond  (at  some  level)  to  the  actual  systems  they 
portray.  The  major  elements  and  interactions  in  a  simulation  should  conform  to 
the  major  elements  and  interactions  in  the  actual  system. 
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•  Present  the  essential  system.  The  simulation  should  contain  all  the  essential 
components,  conditions,  states  and  actions  of  the  entity  it  is  simulating. 

•  Use  appropriate  detail.  The  simulation  should  be  given  a  level  of  detail  that  is 
adequate  for  the  learner. 

•  Make  the  simulation  logical  to  the  user.  Well  designed  instruction  (simulation 
based  or  other)  makes  intuitive  sense  to  students. 

•  Simulations  should  be  inherently  meaningful  to  students.  That  is,  simulations 
should  be  based  on  material  that  explains  how  a  system  operates. 

We  see  then,  that  instructional  simulations  are  usually  conduci\  e  to 
improving  cognitive  structure  when  they  are  matched  with  the  cognitive  level  of 
the  student  and  their  instructional  goals. 

Cognitive  and  Metacognitive  Strategies 

Cognitive  strategies  are  students'  actions  and  thoughts  that  occur  during 
learning  and  that  influence  motivation  and  encoding,  including  acquisition, 
retention,  and  transfer.  Metacognitive  strategies,  on  the  other  hand,  refer  to 
students'  knowledge  about  and  control  over  their  cognitive  processes  (Wittrock, 
1986).  Both  these  terms  are  broad  and  loosely  defined  areas  that  relate  to  many  of 
the  issues  discussed  previously. 

Successful  students  are  found  to  use  a  variety  of  cognitively  oriented 
strategies  to:  (a)  pursue  learning  goals;  (b)  relate  new  knowledge  to  old;  (c) 
monitor  understanding;  (d)  infer  unstated  information;  and  (d)  review, 
reorganize,  and  reconsider  their  knowledge  (Scardamalia,  Bereiter,  McLean, 
Swallow,  &  Woodruff,  1989). 

On  the  other  hand  (and  perhaps  more  importantly),  recent  cognitive 
research  also  indicates  the  ways  unsophisticated  learners  approach  and  think 
about  learning.  Recent  literature,  such  as  Scardamalia,  et  al.  (1989),  indicates 
that  unsophisticated  learners  are  prone  to:  (a)  mentally  structure  information  by 
topic  rather  than  by  goal  (Schoenfield  &  Herman,  1982),  (b)  focus  mainly  on 
surface  features  (Chi,  Feltovich  &  Glazer,  1981;  Silver,  1979),  (c)  use  straight 
ahead  rather  than  recursive  p'  ocedures  (Anzi  &  Simon,  1979),  and  (d)  use  an 
additive  rather  than  transformational  approach  to  learning. 

Implications.  These  findings  indicate  that  in  order  for  instructional 
simulations  to  support  expert-like  (meta)  cognitive  strategies,  they  will  need  to 
encourage  students  to  take  an  active  part  in  the  learning  process,  rather  than  the 
passive  role  they  are  often  assigned  (or  often  assign  themselves).  But  how  can 
such  support  be  provided  ? 

Foremost,  the  simulation  needs  to  provide  the  kind  of  support  that  will 
enable  learners  to  take  maximum  advantage  of  their  own  intelligence  and 
competence.  Following  are  some  suggestions  for  developing  the  kinds  of  supports 
that  are  needed  in  simulation  based  training. 

•  Encourage  strategies  other  than  "practice."  or  "rehearsal." 

•  Support  knowledge  construction  activities. 
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•  Make  concern  for  cognitive  goals  a  very  high  priority.  Make  certain  that 
students  are  aware  of  the  cognitive  goals  that  are  to  be  achieved  by  using  the 
simulation. 

•  Encourage  examination  and  employment  of  existing  knowledge. 

Automaticity  of  Cognitive  Processes 

When  a  task  can  be  performed  such  that  it  no  longer  requires  the  deliberate 
attention  of  the  student,  it  is  said  to  be  automatized.  A  large  body  of  scientific 
research  points  to  the  fact  that  automatic  cognitive  processing  is  a  fast,  parallel 
process,  not  limited  by  short-term  memory,  requiring  little  effort  or  attention  by 
the  performer,  but  requiring  extensive  and  consistent  practice  to  develop.  Its  use 
is  heavily  implied  in  various  aspects  of  skilled  behavior  such  as  reading,  playing  a 
piano,  or  flying  an  aircraft.  In  contrast,  controlled  cognitive  processing  demands 
much  attentional  capacity,  is  serial  in  nature,  is  limited  by  short-term  memory, 
and  frequently  requires  conscious  effort  (Fisk,  1989). 

Implications.  Research  shows  that  for  many  tasks,  both  behavioral  and 
cognitive,  automaticity  develops  slowly  and  often  only  after  massively  repetitive 
practice  (Myers  &  Fisk,  1987).   The  single  most  critical  factor  in  determining  how 
well  a  skill  is  automatized  seems  to  be  the  manner  in  which  the  task  is  presented 
to  the  learner.  As  Fisk  (1989)  stated; 

"Research  supports  the  common  observation  that  the  way  people  typically 
perform  a  novel  task  (i.e.  in  a  slow,  effortful,  and  error-prone  manner)  is 
quite  different  from  how  Jiey  perform  the  task  after  lengthy  practice  (i.e., 
quickly,  accurately,  and  with  little  effort).  Such  performance  differences 
between  novice  and  skilled  individuals  have  led  researchers  to  suggest  that 
complex  performance  is  the  result  of  two  distinct  forms  of  information 
processing.  In  this  paper  we  refer  to  these  processes  as  automatic  and 
controlled  information  processing.  .  .  .  Automatic  processes  can  be 
developed  only  through  extensive  practice  under  consistent  conditions.  .  .  . 
Conversely,  the  use  of  controlled  processes  is  implied  when  no  consistent 
rules  or  no  consistent  sequences  of  information  processing  components  are 
present  in  a  task.  (p.  453-454) 

Automaticity  then  seems  to  be  the  result  of  repeated  exposure  to  consistent 
stimuli  over  a  relatively  short  period  of  time.  In  addition  to  the  admonition  that 
tasks  must  be  consistently  presented  in  order  for  automaticity  to  occur,  several 
other  implications  can  be  gleaned  from  recent  research. 

•  Make  the  consistencies  overt.  Under  certain  conditions,  the  consistent  features 
of  the  simulated  task  may  need  to  be  pointed  out  to  students  so  that  they  can  "tune- 
in"  to  them  (Fisk  &  Schneider,  1981). 

•  Make  the  student  an  active  participant  rather  than  passive  observer. 
Automaticity  implies  active  responding.  The  simulation  should  strive  to  make 
the  student  respond  overtly  to  the  situation  (Schneider,  1985). 
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•  The  difficulty  level  of  the  simulated  task  should  be  such  that  students  are 
consistently  successful  at  performing  it.  Students  may  need  massive  amounts  of 
"successful"  practice  before  the  skill  will  be  automatized. 

•  Training  should  take  place  under  relatively  stress  free  conditions.  A  good 
instructional  strategy  is  to  increase  the  stressfulness  of  the  simulated  task  only 
after  it  is  well  on  its  way  to  being  automatized. 

•  Isolate  components  of  tasks  which  one  wishes  to  automatize.  Frederiksen  and 
White  (1989)  have  shown  that  it  is  possible  to  decompose  cognitive  tasks  and  isolate 
their  components  so  that  automatization  through  training  can  take  place. 

Affect 

The  last  issue  concerns  affect  or  the  motivational  appeal  of  the  instructional 
simulation.  At  first  blush,  affect  may  seem  to  be  fairly  unimportant.  However,  a 
meta-analysis  by  Deckers  and  Donatti  (1981)  indicates  that  while  simulations  may 
not  always  be  more  effective  than  conventional  instruction  methods,  they  exhibit  a 
high  motivational  component.  Earlier  studies  by  Taylor  and  Walford  (1972)  and 
Thiagariajan  (1973)  concur  with  this  finding.  According  to  Adams  (1973),  it  is  a 
simulations  power  to  "unite  the  cognitive  and  affective  areas"  that  makes  it 
potentially  useful. 

Implications.  Affect  is  a  very  complex  construct,  and  as  such,  manifests 
itself  in  many  ways.  For  example  Dittrich's  (1977)  studies  led  him  to  conclude 
that  a  simulation's  perceived  fidelity  is  more  motivational  for  learning  than 
actual  fidelity.  He  suggested  that  either  very  low  or  very  high  fidelity  simulations 
will  be  perceived  as  low  while  medium  fidelity  simulations  will  be  perceived  as 
high  and  hence  will  enhance  positive  motivation  on  the  part  of  students.  Other 
research  shows  that  when  the  (poor)  attitude  of  students  requires  highly 
motivational  instruction,  simulations  incorporating  principles  of  gaming  should 
be  created  (Malone,  1981;  Priestley,  1984;  Reigeluth  and  Schwartz).  Research  by 
Malone  (1981)  indicates  that  three  factors,  challenge,  fantasy,  and  curiosity 
contribute  to  creating  a  positive  affect  on  the  part  of  learners.  The  specific 
guidelines  that  follow  provide  a  brief  summary  of  some  of  the  relevant  literature. 

•  Provide  clear  goals  for  students  to  attain. 

•  Provide  uncertain  outcomes.  If  learners  are  absolutely  certain  to  reach  the  goal 
or  certain  not  to  reach  it,  the  simulation  is  probably  not  challenging  enough  to 
sustain  their  interest. 

•  Use  a  non-zero  based  scoring  system. 

•  Create  competitive  situations. 

•  Stimulate,  satisfy,  and  sustain  student  curiosity. 


Summary  of  Issues  and  Implications 

Table  1  summarizes  the  issues  and  their  implications  for  the  design  and 
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use  of  instructional  simulations.  Many  of  these  implications  are  not  mutually 
exclusive,  but  typically  perform  interactively  during  an  instructional  simulation. 
They  are  set  apart  here  to  emphasize  the  underpinnings  of  the  cognitive 
approach. 

While  these  issues  are  probably  not  new  or  unfamiliar  to  training 
professionals,  it  is  our  observation  that  they  are  not  being  considered  to  the  degree 
that  they  could  be  in  the  design  of  simulation  programs  used  for  training  and 
instruction.  We  believe  that  instructional  simulations  should  be  designed,  used, 
and  evaluated  with  these  implications  in  mind. 


Table  1.  Some  Issues  and  Implications  for  Instructional  Simulations 


Issues 

Implications 

Cognitive  Structure 

Make  simulations  appropriate  for  the  level  students  are  at 

Make  simulations  correspond  to  actual  systems 

Present  only  the  essential  system 

Use  appropriate  detail 

Make  the  simulation  logical  to  the  student 

Make  simulations  inherently  meaningful  to  students 

Cognitive  Strategies 

Encourage  strategies  other  than  "practice"  or  "rehearsal" 

and  Metacognition 

Support  knowledge  construction  activities 

Make  concern  for  cognitive  goals  a  high  priority 

Promote  examination  and  employment  of  existing  knowledge 

Automaticity  of 

Make  consistencies  overt 

Cognitive  Processes 

Encourage  active  participation  rather  than  passive 

Structure  difficulty  to  so  as  to  ensure  consistent  success 

Strive  for  relatively  stress  free  conditions 

Isolate  automatic  components 

Affect 

Provide  clear  goals 

Provide  uncertain  outcomes 

Use  a  non-zero  based  scoring  system 

Provide  a  competitive  situation 

Stimulate,  satisfy,  an'l  sustain  curiosity 
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Introduction:  Perceptions  of  the  Learning  Environment 

This  session's  common  theme  is  that  including  an  "environment  analysis"  in  the 
instructional  development  process  will  improve  learning  system  design.  My  task  is  to  discuss 
the  technology  aspects  uf  such  an  analysis;  however,  what  this  analysis  should  include  is  far 
from  obvious.  Clearly  there  are  several  possible  views  regarding  the  environment's  role  in 
learning:  One,  that  it  is  neutral;  two,  that  it  is  a  relevant  but  ancillary  factor;  and  three,  that 
the  environment  is  an  active  element  in  the  learning  process.  Adopting  the  first  attitude  allows 
the  instructional  designer  to  ignore  the  environment.  Adopting  the  second  mainly  means 
documenting  those  aspects  of  the  environment  that  might  adversely  affect  use  of  an 
instructional  product  in  a  given  setting.  Adopting  the  third  view  means  recognizing  the 
environment  as  part  of  the  technology-strategy-activity  nexus  that  all  teaching  involves. 

I  expect  that  some  might  hold  that  the  environment  is  neutral  with  respect  to  learning; 
after  all,  some  have  argued  that  media  do  not  influence  learning,  that  they  are  mere 
information  carriers,  Kozma  (1991,  p.  205)  argued  that  this  conclusion  was  faulty  because  it 
did  not  take  into  account  the  "cognitively  relevant  characteristics8  of  media.  Time  does  not 
permit  discussion  of  this  issue  here;  the  important  point  is  that  just  as  a  medium  is  more  than  a 
neutral  vehicle,  a  learning  environment  is  more  than  a  neutral  setting  or  locale. 

But  accepting  that  principle  still  leaves  another  question:  Is  the  environment  simply  an 
ancillary  factor  in  instructional  design,  or  might  an  environment,  like  a  medium,  possess 
"cognitively  relevant  characteristics"  that  are,  therefore,  integral  design  factors?  The  way  this 
question  is  answered  obviously  affects  the  ID  process;  therefore  this  inquiry  focuses  on  two 
primary  questions:  1)  What  is  the  case  for  the  premise  that  technological  learning  environments 
are  "active  agents"  in  the  learning  process?,  and  2)  What  are  the  design  implications  of 
accepting  this  premise? 

Aspects  of  A  Technology-Based  Learning  Environment 

An  advertisement  for  an  imaging  system  being  marketed  by  a  major  computer  firm 
proclaimed  that  it  was  "not  just  a  collection  of  graphics  tools  but  a  total  working  visualization 
environment."  Tay  Vaughn  (1988)  defined  the  idea  of  a  computer  "environment,"  saying  that 
the  term  refers  to  "a  set  of  rules,  conditions  and  capabilities  within  which  programs  can  be 
create  J  and  then  run."  The  twin-points  here  are  1),  a  technological  environment  is  more  than 
an  array  of  tools,  it  is  a  working,  function-related,  integrated  system;  and  2),  that  while  a  such 
a  system  may  afford  users  superb  capabilities,  it  also  sets  creative  limits  and  imnoses  ite 
particular  rules  and  regimens  on  the  user. 

Kosma  (1991),  in  his  review  of  media  research,  distinguished  between  the  learner's 
internal  and  external  environments,  and  presented  an  image  of  the  learner  using  his  or  her 
interna!  cognitive  resources  to  manage  the  extraction  of  information  from  the  external 
environment  to  built  new  knowledge.  This  presents  a  markedly  different  view  of  the  learning 
process  from  the  one  on  which  the  "media-are-mere-vehicles"  conclusion  was  based.  Also,  it 
reminds  instructional  developers  that  anything  that  affects  information  acquisition  and  use  is  an 
active  element  of  the  learner's  "external  environment." 


John  Sculley  (1991)  provided  an  interesting  perspective  on  learning  environments 
while  addressing  the  problem  that  although  modern  technology  has  greatly  improved 
information  processing  capabilities,  white  collar  productivity  has  not  similarly  improved,  either 
in  schools  or  in  the  workplace.  Sculley  said  this  was  because  "new  technology  has  been 
mapped  on  top  of  an  old  work  model  and  an  old  learning  model. "  In  the  broadest  sense,  then, 
an  instructional  environment  is  what  results  from  mapping  a  technology  model  on  top  of  a 
learning  model.  Every  learning  environment  thus  embodies  what  Martin  Tessmer  calls  a 
"technology  culture"  comprised  of  the  available  technology,  the  attitudes  and  expertise  of  the 
learners  and  teachers  who  will  use  it,  and  their  perceptions  of  how  it  ought  to  be  used  to 
facilitate  learning.  Thus  it  is  reasonable  to  conclude  that  learning  environments  do  possess 
"cognitively  relevant  characteristics N  that  must  be  addressed  during  design. 

The  environment  and  learning  question,  then,  breaks  down  into  three  related 
questions:  What  is  the  "new"  technology  model?  What  is  the  HnewH  education  model  that 
technology  is  supposed  to  implement?  And,  how  should  they  be  connected?  Examining  the 
old  models  will  facilitate  understanding  the  new  models. 


A  technology  model  embodies  two  complementary  ideas--an  image  of  tool-use  th?.t 
links  device  and  user  in  purposeful  activity,  and  an  image  of  some  functional  goal  that  puts  the 
"purpose"  in  that  activity.  For  centuries,  the  printed  page  has  been  the  basic  tool  for 
managing  information,  and  Sculley  says  that  even  through  the  1980s  it  remained  as  the 
"computing  metaphor  for  documents/  Early  in  this  century,  Thorndike  provided  a  "big  idea" 
about  using  the  print  medium  as  an  instructional  technology  when  he  wished  for  a  device  that 
would  not  allow  the  student  to  go  to  page  two  until  he  mastered  what  was  on  page  one.  No 
one  did  much  with  that  idea  for  a  long  time,  but  this  image  of  tool-use  remained  until  it 
emerged  in  the  1960s  as  the  essential  idea  underpinning  programmed  instruction. 
Although  programmed  instruction  as  a  media  form  did  not  last,  the  basic  notions  that  came  out 
of  that  tradition  remain  at  the  center  of  today's  dominant  instructional  paradigm.  I  call  it  the 
tools  and  tasks  paradigm  because  it  invokes  a  particular  image  of  media  use  and  teaching:  The 
basic  learning  environment  is  the  classroom;  the  teacher  controls  the  use  of  technology;  the 
learner  is  subordinate  to  the  teacher;  media  are  regarded  as  simple  delivery  tools;  the  objective 
is  assimilation;  and  the  value  that  drives  evaluation  is  efficiency. 


Sometime  after  Thorndike  offered  his  suggestion  for  a  model  teaching  tool,  another 
big  idea,  attributed  to  Frederick  Taylor,  took  hold.  This  was  the  mass  production-based 
"factory  model"  which,  Sculley  (1991)  says,  separated  planning  from  doing  and  reduced  the 
worker's  role,  thus  minimizing  the  knowledge  and  skill  required.  Education's  didactic  form 
reflected  this  model,  and  didactic  instruction,  though  widely  criticized,  remains  the  dominant 
form  of  teaching.  Its  deficits  have  been  cataloged  by  Brown,  Collins  and  Duguid  (1989), 
among  others:  Knowledge  is  viewed  as  a  quantity  of  abstract,  self-contained  facts  and  concepts 
to  be  "delivered"  to  the  learner.  The  context  of  learning  is  thought  to  be  "merely  ancillary"  to 
the  learning  process  and  no  serious  thought  is  given  to  how  the  learning  situation  might  affect 
cognition  or  to  how  to  make  learning  activities  "authentic."  Thus  knowledge  is  separated  from 
its  environment-of~origin  and  no  link  between  learning  and  use  is  formed.  Consequently, 
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success  in  the  school  setting  does  not  ensure  success  in  an  application  setting  where  problems 
are  not  always  well-defined,  nor  solutions  always  clear-cut. 


The  "New"  Technology  Model 

Over  a  decade  ago  Manfred  Kochen  (1981)  saw  the  convergence  of  computers  and 
communications  as  a  new  technology  in  the  making  (p.  148).  Since  then,  powerfiil  personal 
computers  have  become  ubiquitous,  optical  disc  systems,  fiber  optics  and  networking 
technology  have  all  advanced  enormously.  Computer-controlled,  optical  disc  systems 
epitomize  the  new  technology.  Such  systems  easily  surpass  older  media  in  technical 
capabilities  and  also  hold  much  potential  for  changing  the  learning  process.  This  potential  will 
be  lost,  however,  if  the  technology  is  used  with  a  deficient  learning  model. 


The  "New"  Learning  Model 

While  there  is  no  definitive  "new"  learning  model,  several  presently  emerging  themes 
illustrate  a  new  line  of  thinking  about  instruction.  One  of  these  themes,  as  explained  by 
Brown,  Collins  and  Duguid  (1989),  is  called  "situated  cognition."  This  idea  challenges  the 
notion  that  the  environment  or  situation  in  which  learning  occurs  is  "neutral"  or  "ancillary" 
and  argues  that  knowledge  is  a  product  of  context,  activity  and  culture  (p.  42),  that  learning 
and  cognition  "are  fundamentally  situated. "  They  see  conceptual  knowledge  not  as  an  abstract 
entity  but  as  a  tool,  and  artfie  that  just  as  one  acquires  skill  in  using  a  tool  through 
engagement  in  "authentic  activities"  in  real-world  situations,  conceptual  knowledge  should  be 
developed  through  student  work  that  reflects  what  practitioners  in  the  domain  actually  do. 

A  major  design  idea  of  the  situated  cognition  perspective  is  that  problem  solving  tasks 
should  be  staged  so  that  a  solution  is  reached  not  just  by  rote  processing  but  by  means  that 
enable  the  learner  to  use  the  environment  to  solve  the  problem,  means  that  are  said  to  be  "in 
conjunction  with  the  environment"  (Brown,  Collins  and  Duguid,  1989,  p.  35). 

A  related  idea  called  "anchored  instruction"  is  under  study  by  The  Cognition  and 
Technology  Group  at  Vanderbilt  (1991).  This  group  is  doing  research  on  "the  effects  of 
situating  instruction  in  videodisc-based,  problem  solving  environments"  (p.  2)  which  are 
intended  to  permit  students  to  engage  in  "sustained  exploration"  of  a  problem  from  "multiple 
perspectives"  (p.  3). 

In  another  context,  Rappaport  and  Halevi  (1991,  p.  69)  offer  an  interesting  thesis 
about  the  computer  industry  when  they  predict  that  by  the  year  2000,  "the  most  successful 
computer  companies  will  be  those  that  buy  computers  rather  than  build  them."  As  a  current 
example,  they  cite  Microsoft,  a  company  that  does  not  manufacture  computers,  but,  as  the 
leading  developer  of  microcomputer  software,  has  become  more  successful  than  anyone  who 
does.  What  does  that  have  to  do  with  instructional  development?  The  essential  idea  is  that  the 
software  that  Microsoft  makes  does  more  than  merely  allow  computers  to  run,  it  defines  the 
"desktop  computing  environment"  (p.  72),  and  the  nature  and  value  of  computing.  Likewise, 
good  instructional  d^;  n  does  more  than  merely  use  computers  and  media  well,  it  defines  the 
learning  environment    value  and  utility.    Sculley  (1991),  speaking  about  productivity 
generally,  says:  "The  real  challenge  for  developers  will  be  to  create  products  based  on 
emerging  technologies  that  reflect  the  changing  way  that  people  work  in  an  information 
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economy  (p.  9).  Instructional  developers,  I  believe,  could  adopt  this  challenge  without 
changing  a  word. 


Forging  the  Technology-Cognition  Connection 

As  the  problem  of  finding  powerful  technology  to  apply  to  instructional  problems 
becomes  secondary  to  learning  how  to  use  that  technology  to  achieve  the  larger  goals  of 
reorganizing  work  and  learning,  successful  instructional  development  will  require  added 
emphasis  on  the  design  of  learning  environments  that  take  into  account:  One,  principles  of 
Mhuman  factors,"  especially  human-computer  interaction;  two,  learner  involvement  strategies; 
three,  the  "situated  nature"  of  learning,  and  four,  the  problem  of  providing  "authentic" 
learning  situations. 

Human  Factors 

Human  factors  considerations  relate  to  environment  analysis  at  several  levels. 
Ergonomics  deals  with  the  physical  aspects  of  using  technical  devices.  But  human  factors  aiso 
must  address  those  human  capabilities  and  limit jfions  that  affect  cognition  and  learning 
(Staggers,  1991,  p.  47). 

Some  very  useful  ideas  on  the  application  of  human  factors  to  instructional 
development  are  to  be  found  in  a  recent  paper  by  Orey  and  Jih  (1991).  They  contend  that  the 
nature  of  interaction  between  computers  and  human  users  is  a  "neglected  area"  of  study,  and 
they  list  several  techniques  now  being  explored  to  facilitate  employing  human  factors  in 
instructional  design.  These  include  adaptive  interfaces,  task  analysis,  measuring  user's  mental 
models,  design  audits,  mock-ups,  prototyping,  and  field  trials. 

All  are  intended  to  make  human-computer  interaction  effective  and  "user-friendly,"  based  on 
the  awareness  that  in  any  setting,  student  populations  will  include  people  who  possess  vastly 
different  skill  in  using  computers. 

Perhaps  the  most  interesting  element  in  their  paper  is  a  chart  that  relates  human 
factors  guidelines  to  each  of  Gagne's  nine  events  of  instruction*  a  widely-used  construct  ir 
instructional  development. 

There  is  one  research  study  with  human  factors  elements  that  I  find  very  interesting. 
An  English  professor  at  the  University  of  Delaware,  Marcia  Peoples  Halio,  did  a  five-year 
study  in  which  she  compared  the  quality  of  student  papers  produced  on  Macintosh  and  MS- 
DOS  machines.  As  reported  in  the  Washington  Post  by  T.  R.  Reid  (1991),  she  found  that 
DOS  users  produced  papers  with  much  higher  readability  scales  and  with  far  fewer  spelling 
errors.  Moreover,  she  also  noted  that  DOS  users  were  more  likely  to  write  about  serious 
issues,  such  as  war  and  pollution,  while  MAC  users  wrote  about  such  topics  as  fast  food. 

The  big  question,  of  course,  is,  what  does  this  mean?  Reid's  conclusion  was  simple: 
"Mac  people"  are  different  from  "DOS  people."  IBM  people,  he  says  are  "no-nonsense"  types 
who  follow  established  ways  of  doing  things  while  Mac  people  are  "free  spirits"  and  "artistic 
types"  who  worry  less  about  rules  of  grammar  and  spelling.  Thus  if  "Mac  people"  are  seen  as 
a  more  creative  lot  and  "DOS  people"  as  a  more  serious  group,  it  is  interesting  to  speculate 
what  will  happen  as  the  Mac  becomes  more  like  an  IBM-PC  and  DOS-PCs  adopt  Macintosh- 
like  user  interfaces. 
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Learner  Involvement 


Implicit  in  any  learning  environment  design  is  a  particular  "image  of  the  learner " 
(Ullmer,  1989)  that  defines  how  the  learner  is  expected  to  use  that  environment.  One  such 
"image*  is  what  Alan  Kay  (1991,  p.  138)  calls  the  "fluidic  theory  of  education"  which  holds 
that  students  are  "empty  vessels"  to  which  k  lowledge  is  to  be  tranrferred  "drop-by-drop." 
The  notion  that  media  are  mere  delivery  vehicles  draws  heavily  on  this  image.  Not 
surprisingly,  Kosma's  (1991,  p.  179)  analysis  of  media  research  is  based  on  an  entirely 
different  image-that  of  a  learner  "actively  collaborating"  with  the  environment  "to  construct 
knowledge/  and  strategically  managing  his  or  her  "cognitive  resources"  to  create  new 
knowledge. 

James  Dezell  (1988,  p.  10)  of  IBM  says  that  technology-based  learning  environments 
should  be  "responsive  to  the  ways  today's  children  receive  information"  and  that  these 
environments  should  bi*  "programmed  for  discovery  rather  than  instruction." 

Kay  (1991,  p.  13$)  picks  up  on  the  same  theme  but  carries  it  a  bit  further.  He 
believes  that  powerful,  networked  computers  can  greatly  enhance  learning,  "but  only  in  an 
educational  environment  that  encourages  students  to  question  facts  and  seek  challenges,"  to  not 
accept  everything  at  face  value.  He  agrees  with  Jerome  Bruner  that  people  can  devise  for 
themselves  new  ways  of  thinking  and  thus  expand  their  understandings. 

Situated  Learning 

The  main  idea  behind  situated  learning,  to  repeat,  is  that  environments  structure 
cognition,  that  situation,  activity  and  culture  co-produce  knowledge. 

Kosma  (1991,  p.  180)  holds  that  the  learning  process  is  "sensitive  to  characteristics 
of  the  external  environment  such  as  the  availability  of  specific  information  at  a  given  moment, 
the  duration  of  that  availability,  the  way  the  information  is  structured,  and  the  ease  with  which 
it  can  be  searched. "  Thus  these  become  important  design  consideration  with  regard  to 
mediating  instruction  in  any  learning  environment. 

Brown,  Collins  and  Duguid  (1989)  provide  a  very  interesting  example,  drawn  from 
Lave's  work,  of  how  a  situation  can  help  structure  cognition.  The  example  is  from  a  Weight 
Watchers  class  and  the  problem  was  how  to  create  a  cottage  cheese  serving  thaf  equalled  thiee- 
fourths  of  the  two-thirds  cup  that  the  program  allowed.  They  relate  how  the  problem  solver 
paused  to  size-up  the  situation,  then  suddenly  proceeded  to  fill  a  measuring  cup  two-thirds  full, 
dumped  it  onto  a  cutting  board,  patted  it  into  a  circle,  marked  a  cross  on  it,  removed  one 
quadrant,  and  served  the  rest.  This  is  problem  solving  carried  out  "in  conjunction  with  the 
environment." 

Authentic  Situations 

Alan  Kay  (1991,  p,  138)  relates  an  observation  by  physicist  Murray  Gell-Mann  that 
compares  modem  education  to  being  taken  to  a  fine  restaurant  ami  being  fed  the  menu.  He 
meant  that  students  are  taught  in  a  superficial  manner  in  which  representations  of  ideas  replace 
the  ideas  themselves*  Technology  will  soon  provide  students  with  computerized  access  to 
massive  date  resources.  The  critical  design  question,  Kay  says,  is  how  to  get  from  the  menu 
to  the  food.  Put  another  way,  how  can  designers  create  the  authentic  environments  that  enable 
students  to  acquire  meaningful,  usable  knowledge? 
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The  example  of  the  Weight  Watchers  class  illustrates  the  value  of  providing  an 
authentic  situation  in  which  problem  solving  is  carried  out  in  conjunction  with  the 
environment.  But  obviously  teachers  cannot  issue  cottage  cheese  and  other  needed  items  to 
everyone  studying  fractions.  Nor  can  medical  student:  be  given  open  use  of  laboratories  and 
expensive  diagnostic  equipment  and  be  issued  a  patient  with  a  designated  illness.  This  is 
where  technology  is  most  useful-in  providing  realistic  simulations  of  the  types  of  problems 
that  practitioners  actually  encounter.  But  designers  have  to  be  able  to  recognize  the  key 
elements  of  the  environment  to  be  simulated. 

One  excellent  example  of  such  a  simulation  is  the  National  Board  of  Medical 
Examiners  Computer  Based  Testing  or  CBX  program.  The  program  includes  several  case 
simulations  in  each  of  several  medical  areas.  A  case  might  begin  with  a  patient  in  an 
emergency  room.  The  student  user  takes  a  patient  history  and  does  a  physical  exam  of  varying 
completeness  depending  on  the  urgency  of  the  problem.  The  user  then  prescribes  appropriate 
therapeutic  measures.  The  program  is  realistic  in  that  the  user,  to  obtain  the  results  of  tests 
ordered,  must  advance  the  "clock/  and,  as  the  clock  advances,  the  disease  also  progresses. 
As  laboratory  results  are  returned,  the  user  adjusts  the  treatment  regimen  as  deemed 
appropriate.  Once  the  course  of  the  patient  has  been  established  and  the  simulation  ends,  the 
user  receives  a  list  of  procedures  ordered  correctly,  those  ordered  unnecessarily,  and  decisions 
made  that  were  harmful  to  the  patient  (Wilkes,  1990). 

Numerous  other  simulations  that  employ  similar  principles  are  presently  being  used  in 
health  sciences  education  and  educators  are  gaining  experience  in  their  design,  and  in 
discovering  the  basic  requirements  of  good  design. 

Dertouzos  (1991)  provides  an  interesting  way  of  looking  at  specifying  design 
requirements  for  infomiation  technologies.  To  escape  "the  present  chaos"  regarding  computer- 
based  networks,  and  to  fashion  them  into  "a  true  information  infrastructure,"  he  says  networks 
must  have  three  key  capabilities:  flexible  information  transport  capabilities,  common  services 
and  common  communications  conventions"  (p.  65).  Because  computer  network  users  have 
varying  needs  regarding  transmission  speed,  reliability  and  security,  and  because  it  is 
expensive  to  maximize  all  three,  Dertouzos  suggests,  metaphorically,  that  it  would  be  ideal  if 
users  could  set  "levers"  to  receive  the  desired  level  of  each  capability  as  needed.  Now  speed, 
security  and  leliability  are  important  in  instructional  systems,  too,  but  other  design  factors  may 
be  more  important  in  creating  authentic  learning  environments. 

The  question  is:  What  are  they?  What  "levers"  does  a  designer  need  to  set  to 
maximize  the  effectiveness  of  an  instructional  progiam? 

Two  levers  are  relevant  to  specifying  the  nature  of  the  problem:  one  to  make  the 
simple-to-complex  setting,  and  one  to  make  the  fixed-solution  to  open-solution  setting. 

Two  levers  also  are  needed  to  set  situation  authenticity.  Clyman  (1991)  outlined 
several  assumptions  that  underpin  the  design  of  the  CBX  system  discussed  earlier.  One  is  that 
the  system  will  provide  the  degree  of  realism  needed  to  elicit  the  behavior  of  interest  from  the 
student,  and  the  second  is  that  the  system  will  "capture  those  behaviors"  in  a  form  that 
reasonably  represents  those  behaviors.  So  one  lever  varies  the  degree  of  realism  the  problem 
presentation  provides,  the  second  addresses  the  problem  of  capturing  what  the  student  really 
wants  to  do. 

Kay  (1991)  offered  an  interesting  insight  on  what  he  considers  "a  well -conceived 
environment  for  learning."  It  should  "be  contentious,  even  disturbing,"  it  should  "seek 
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contrasts  rather  than  absolutes,  aim  for  quality  over  quantity,  and  acknowledge  the  need  for 
will  and  effort"  (p.  140).  This  identifies  two  additional  levers:  one  to  set  the  degree  of 
contentiousness,  and  the  second  to  adjust  the  balance  between  knowledge  quality  and  quantity. 

An  observation  about  the  new  software  form  called  "groupware"  provided  another 
example  in  which  a  personality  trait  was  attributed  to  a  computer  system.  Because  some  users 
resented  the  high  degree  of  control,  a  software  program  was  redesigned  to  bo  "less  strident. " 
This  suggests  another  lever  to  adjust  the  stridency  level  of  a  teaching  program. 

Any  simulation  design  will  likely  involve  these  basic  assumptions  and  problems. 
Instructional  developers  are  taught  well  to  follow  rational  design  models.  But  building  an  ideal 
learning  environment  arguably  requires  something  more— in  applying  human  factors,  and  in 
setting  the  "levers"  of  design. 
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Introduction 

Interactive  video  may  be  conceived  as  the  combination  of  the  interactive 
capabilities  of  the  computer  and  the  audiovisual  power  of  video.  This 
combination  is  generally  considered  as  a  powerful  medium  for  delivering 
instruction,  nowadays  often  incorporated  in  so-called  multi-media  systems.  This 
article  focuses  on  one  design  issue  with  respect  to  the  video  component. 

The  Problem 

An  essential  property  of  adding  computer  control  to  video  programs  is  the 
possibility  to  stop  for  questions  or  excercises,  or  to  givr  feedback  to  students 
about  their  progress,  at  any  desired  moment.  A  simple  but  fundamental  question 
is:  What  is  the  optimum  moment  to  stop  for  the  purpose  of  achieving  maximum 
learning?  The  literature  shows  some  ideas  with  respect  to  this  question  (see  for 
instance  Bork,  1987;  Laurillard,  1987;  Phillips,  Hannafin  &  Tripp,  1988;  SchafTer 
and  Hannafin,  1986),  One  question  is  whether  preferred  segment  length  plays  a 
role  in  this.  This  paper  attempts  to  clarify  this  issue  in  the  context  of  a  larger 
study  about  optimal  segment  length  (Verhagen,  in  preparation).  To  make  the 
research  manageable,  choices  were  made  that  narrow  the  problem  of  that  study 
down  to  the  following  central  question: 

"  What  is  the  optimum  length  of  well  designed  audiovisual  segments  to  present 
factual  information  via  an  interactive  video  program  to  learners  who  possess 
certain  characteristics?" 

The  exact  meaning  of  this  question  is  specified  as  follows: 

-  The  measure  for  "length  of  segment"  is  the  number  of  information  elements  in 
one  segment.  The  term  "information  element"  is  defined  as  the  smallest 
meaningful  unit  of  language  to  state  a  name,  a  label  or  a  single  proposition, 
according  to  Gngnes  definition  of  verbal  informatiot  (Gagn6,  1985). 

-  "Well  designed"  is  operationnlized  in  terms  of  the  approval  of  the  audiovisual 
material  by  an  expert  jury,  in  combination  with  successful  formative  testing  of 
the  material  under  development. 
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-  "Factual  information"  neans  verbalizable  information  about  facts,  concepts, 
procedures  and  principles  to  be  stored  in  memory  on  the  recall  level  (see  also 
Gagm*,  1985;  or  Merrill,  1983;  Romizowsk;,  1986;  Wager  &  GagnS,  1988). 

•  The  learner  characteristics  involved  concern  verbal  ability;  the  ability  to  process 
pictorial  information;  a  cognitive  style  aspect  (field  dependance);  and  sex.  These 
are  briefly  described  below. 

One  term  cannot  yet  be  defined,  and  that  is  what  to  understand  by  "optimum" 
length  of  a  segment.  It  is  part  of  the  study  to  find  out  what  has  to  be  regarded  as 
optimum.  In  this  paper,  one  aspect  is  elaborated:  the  extent  to  which  learner 
preference  influences  the  way  in  which  optimum  segment  length  has  to  be 
regarded. 

With  respect  to  recall,  two  situations  have  to  be  taken  into  account: 

a.  Segment  length  and  cjirect  recall  of  information 

In  general,  video  segments  serve  to  initially  supply  learners  with  information 
as  a  part  of  a  learning  experience  that  may  further  consist  of  rehearsal  or 
application  of  that  information,  with  or  without  the  demand  to  demonstrate 
the  ability  to  recall  the  presented  information  by  answering  questions.  The 
ability  of  learners  to  recall  the  presented  information  directly  after  the 
completion  of  a  video  segment  can  be  considered  as  a  measure  of  the 
effectiveness  of  the  initial  presentation.  The  better  this  is  the  case,  the 
quicker  a  learner  may  proceed  in  an  instructional  situation  and  the  shorter 
the  needed  time  on  task  will  be  and  thus  the  more  efficient  the  learning  will 
take  place,  at  least  as  far  as  concerns  this  factor.  This  efficiency  may  be 
dependent  on  segment  length. 

b.  Segment  length  and  delayed  recall  of  information 

After  a  learner  has  worked  through  all  segments  of  a  program,  the 
effectiveness  of  the  learning  session  as  a  whole  may  be  measured  from  his  or 
her  ability  to  answer  questions  about  the  program  after  some  time  has 
elapsed.  This  effectiveness  may  also  appear  to  be  dependent  on  segment 
length. 

The  relation  between  segment  length  and  direct  recall  may  or  may  not  tally  with 
the  relation  between  segment  length  and  delayed  recall.  On  one  hand,  correct 
direct  recall  may  be  a  proof  of  successful  information  transfer,  which  may  be 
confirmed  by  a  high  level  of  delayed  recall.  On  the  other  hand;  in  a  learning 
situation,  mistakes  with  respect  to  direct  recall  will  in  most  cases  be  remediated 
by  rehearsing  the  missed  information  in  one  form  or  another.  Rehearsal  supports 
retention.  Incorrect  direct  recall  may  thus  cause  rehearsal  which  leads  to  better 
delayed  recall  than  correct  direct  recall,  be  it  at  the  expense  of  increased  time  cn 
task.  However,  there  is  reason  to  strive  for  maximum  correct  direct  recall: 
rehearsal  as  a  consequence  of  the  frustrating  experience  of  failing  to  answer 
questions  correctly  violates  rules  for  motivational  instruction,  Keller  for  instance 
argues  that  instruction  should  be  such  that  learners  can  develop  confidence  in 
their  possibilities  to  fulfill  assigned  learning  tasks  successfully  (see  Keller,  1983; 
Keller  &  Kop,  1987;  Keller  &  Suzuki,  1988).  Rehearsal  should  therefore  not  be 
the  consequence  of  failure  by  the  learner,  but  should  be  carefully  planned  and 
integrated  in  the  learning  situation  ns  an  activity  that  appears  logical  to  the 
student  and  may  thus  be  easily  accepted.  It  is  possible  that  preferred  segment 
length  automatically  provides  the  answer  to  this  problem,  if  learners  are  in  the 
position  to  make  themselves  the  trade-off  between  segment  length  and  failure  to 
answer  questions  about  the  presented  information,  The  experiment  that  is 
decribed  below  is  designed  to  study  this  possibility. 


9 

ERJC 


8SU 


PRKKKRRKD  LENOTII  OP  VIDEO  RtitiMKNTH  IN  INTKKACTIVK  VIDBO  PIUXJRAMS 


PACJB  3 


Before  turning  to  a  description  of  the  experiment  it  has  to  be  emphasized  that  the 
study  has  a  non-instructional  character.  The  study  is  primarily  designed  to  find 
rules  about  segment  length  for  the  purpose  of  the  initial  presentation  of 
information  as  a  component  of  one  instructional  pattern  or  another.  The  role  of 
such  presentations  within  instructional  patterns  is  in  principle  not  considered.  It 
was  however  attempted  to  support  the  validity  of  the  segments  used  in  the 
experiments  as  potential  building  blocks  in  instruction  by  using  a  pattern  of 
presentation,  questioning,  remediation,  and  feedback  that  resembles  tutorial 
instruction. 

Self-chosen  segment  length  is  one  possibility,  program-controlled  segment  length 
another.  The  desirability  of  self-chosen  segment  length  has  to  be  valued  against 
its  effectiveness  for  learning  compared  to  program-determined  fixed  segment 
length. 

A  further  point  is  whether  segment  length  can  be  independent  of  time  on  task  in 
the  sense  that  long  learning  sessions  may  cause  fatique  effects  that  affect  optimal 
segment  length  (after  some  time  students  may  probably  need  shorter  segments 
than  when  they  are  fresh). 

With  the  former,  the  following  research  questions  are  listed: 

1.  What  is  the  preferred  segment  length  if  learners  have  to  decide  for  themselves 
how  much  information  (how  many  information  elements)  they  want  to  have 
presented  to  them  before  they  stop  to  answer  questions  about  that 
information? 

2.  Which  is  the  relationship  between  segment  length  and  direct  recall  of  factual 
information  presented  by  these  segments 

a,  when  segment  length  is  self-chosen? 

b.  when  segment  length  is  fixed? 

3.  Which  is  the  relationship  between  segment  length  and  delayed  recall  of 
factual  information  presented  by  these  segments 

a.  when  segment  length  is  self-chosen? 

b.  when  segment  length  is  fixed? 

4.  D^es  fatigue  effect  affect  self-chosen  segment  length? 


Research  Method 

In  the  study,  subjects  attended  two  experimental  sessions.  During  the  first,  they 
mainly  worked  with  an  experimental  videodisc  program,  followed  by  a  posttest. 
This  session  took  one  morning  or  one  afternoon.  The  second  session  was  used  for 
a  retention  test  and  for  tests  to  score  some  variables  with  respect  to  individual 
characteristics.  The  second  session  took  about  two  hours. 


The  experimental  video  program, 

For  the  study,  an  experimental  videodisc  program  about  cheesemaking  has  boon 
produced.  This  program  contains  252  information  elements  that  are  presented 
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through  video  with  offscreen  narration.  Together  they  form  a  connected 
discourse  of  36  minutes  if  the  program  is  played  linearly  without  stopping. 
According  to  the  earlier  definition,  an  information  element  is  operationalized  as 
one  uninterrupted  statement  of  the  narrator  about  which  one  factual  question 
can  be  put.  This  means  that  in  the  experiments  with  one  question  per 
information  element  in  principle  252  questions  could  be  asked  about  the  content 
of  the  program.  However,  in  eight  cases  this  would  have  been  too  artificial 
because  of  the  simplicity  of  the  text  concerned.  In  practice  244  questions  were 
put  to  the  subjects.  The  252  information  elements  are  distributed  over  150 
camera  positions  (shots). 

The  computer  program  is  used  to  control  the  system  so  that  the  video  program 
can  be  broken  down  into  various  sequences  of  segments.  Thereby  the  video  can 
only  be  started  or  stopped  between  two  information  elements .  This  causes  video 
segments  of  the  program  always  to  consist  of  whole  numbers  of  information 
elements.  The  way  in  which  the  program  is  broken  down  into  segments  depends 
on  the  different  experimental  conditions  that  are  described  below  . 

The  video  program  has  two  parts:  (a)  an  introduction  of  4.5  minutes  (information 
elements  1-33;  31  questions),  and  (b)  a  main  program  of  31.5  minutes 
(information  elements  34-252;  219  elements  with  213  accompanying  questions). 

In  all  conditions,  the  introduction  was  used  twice.  The  first  time  the-  introduction 
was  presented  as  an  ordinary  linear  video  prograr,  tc  give  an  overview  of  the 
cheesemaking  process.  This  prepares  the  subjects  for  the  presentation  >tyle  and 
the  kind  of  information  in  the  main  program  (in  which  the  cheese?taaking  ptocess 
is  treated  in  much  more  detail). 

The  second  time  the  introduction  was  used  to  give  the  subjects  the  opportunity  to 
experience  the  way  of  working  with  the  main  program  according  to  the 
experimental  condition  in  which  he  or  she  was  placed. 

It  is  believed  that  by  this  approach  the  subjects  were  well-prepared  to  enter  the 
main  program  (all  data  about  working  with  the  program  were  collected  during 
work  with  the  main  program). 

Experimental  conditions 

Five  experimental  conditions  were  used: 

VAR: 

In  the  so-called  variable  condition,  the  subjects  determined  the  length  of  each 
segment.  The  basic  procedure  was  as  follows: 

-  In  the  ready  position  to  begin  a  new  segment,  the  screen  showed  a  video  still 
with  the  superimposed  message  "click  the  mouse'1.  This  video  still  was  the  first 
frame  of  the  information  element  with  which  the  segment  would  start. 

-  After  clicking,  the  program  started  playing  until  the  subject  decided  that  it  was 
time  to  ask  for  it  to  stop  by  again  clicking  a  mouse  button.  The  program  then 
stopped  at  the  end  of  the  information  element  in  which  the  stop  was  requested. 
This  defines  a  segment:  if  the  playing  started  with  element  i,  and  the  program 
stopped  at  the  end  of  element  i+k,  a  segment  containing  k+1  information 
elements  was  created. 
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-  The  subject  then  answered  all  k+1  questions  about  the  segment  he/she  just  saw, 
one  question  after  the  other,  All  questions  were  open  questions  that  required 
short  sentences,  sir»qle  words,  or  numbers  to  be  typed  in  as  answers. 

-  Next,  all  questions  ..c/v  one  by  one  repeated  corresponding  to  the  order  of  the 
information  elements  in  the  segment  together  with  the  answers  of  the  subjects 
and  feedback  about  right  or  wrong.  Right  answers  were  reinforced  by  a 
complete  sentence  that  restated  the  correct  answer.  If  a  question  was 
encountered  that  was  answered  wrongly,  the  feedback  informed  the  subjoct  of 
this  fact  followed  by  the  message  "click  the  mouse  to  repeat  the  related  video 
fragment".  After  clicking,  this  video  fragment  was  repeated  followed  by  a 
second  attempt  to  answer  the  question.  Also  after  this  second  time  feedback 
aboul  right  or  wrong  was  provided,  this  time  together  with  the  right  answer 
regardless  of  wh^t.ier  the  second  answer  was  right  or  wrong. 

-  After  all  questions  of  the  segment  were  reviewed,  the  subject  was  allowed  to 
continue  the  program  beginning  with  element  i+k+1,  the  information  element 
that  folJowed  the  last  element  of  the  just-completed  segment1. 

In  this  way,  the  subject  divided  the  video  program  into  segments  by  the  repetition 
of  watching,  stopping,  answering  questions,  getting  feedback  (with  built-in 
repetition  of  video  parts  with  respect  to  missed  question?),  and  starting  the  next 
segment. 

The  VAR  condition  is  the  main  condition  used  to  answer  the  research  questions 
about  preferred  segment  length  (Research  Questions  1,  2a  and  3a). 

CROSSED: 

The  same  as  VAR,  with  the  difference  that  first,  the  second  half  of  the  main 
program  was  presented  (information  elements  145-252),  and  then  the  first  half 
(information  elements  34-144).  This  was  possible  because  the  subjects  entered 
the  main  prorjvam  after  the  introduction  (with  the  overview  of  the  cheesemaking 
process)  had  been  studied  twice.  With  that  background  it  was  not  an  unnatural 
experience  to  start  in  the  middle  of  the  main  program. 

This  condition  was  used  to  find  out  whether  fatique  effects  would  affect  the 
results  (Research  Question  4). 

LIN: 

In  this  condition  no  stopping  was  allowed.  The  subjects  watched  the  complete 
main  program  without  stopping  and  then  had  to  answer  the  213  questions  of  the 
main  program  as  if  they  had  divided  the  program  into  just  one  long  segment. 
This  condition  was  made  to  compare  linear  use  of  the  video  program  with 
interactive  use.  ft  was  meant  as  a  control  condition  relative  to  information 
overload  if  the  experimental  program  was  not  segmented, 

SHORT-LONG  (SL): 

In  this  condition,  segment  length  was  fixed.  The  first  half  of  the  program  was 
worked  through  in  23  steps  (short  segments),  the  S"  ond  half  in  5  steps  (long 
segments).  Except  for  fixed  length,  this  condition  functioned  in  the  same  way  as 

h      At  thin  place    where  n  segment  just  hns  been  completed  -  subjects  could  also  bnve  a  break  nnd 
leave  their  choirs  to  relnx  in  nn  ndjncenl  room.  To  do  so  they  were  required  to  click  the  right- 
hnnd  mouse  button,  which  caused  the  message  "pause"  to  be  displayed  nnd  by  which  the  logging 
of  time,  was  switched  to  the  logging  of  the  durntion  of  the  interval,  To  continue  afler  a  pause, 
subjects  had  to  hit  <ENTKU>  after  which  the  video  still  of  the  first  element  of  the  next  segment 
was  displayed  together  with  the  message  "click  the  mouse"  as  described  above. 
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the  VAR  condition.  The  SL  condition  was  used  to  show  how  recall  of  facts  was 
affected  by  fixing  the  segment  length  (Research  Questions  2b  and  3b). 

LONG  SHORT  (LS): 

The  same  as  the  former,  with  the  difference  that  in  this  case  the  long  segments 
came  first  This  condition  was  made  to  balance  the  former. 


Subjects 

Subjects  were  235  univer?;ty  freshmen  of  several  technical  and  social  science 
departments  at  a  single  v  diversity  in  the  Netherlands.  Age,  sex,  university 
department,  and  year  of  study  were  registered.  The  subjects  were  randomly 
distributed  over  the  experimental  conditions  with  the  restriction  that  in  the  first 
phase  of  the  experiment,  onSy  the  conditions  UN,  VAR  and  CROSSED  existed. 
Students  of  the  Department,  of  Education  participated  in  that  phase  only.  In 
total,  3  :>ubu*ctn  were  removed  from  the  data  analysis  because  of  false  data  due  to 
obvious  atypical!  behaviour  or  as  a  consequence  of  technical  failures  of  the 
equipment  during  the  experiment.  The  resulting  distribution  is  presented  in 
Table  1. 

Table  I:  Distribution  of  the  subjects 


UN 

VAK 

CROSSED 

LS 

SL 

Total 

TO: 

22 

27 

22 

0 

0 

71 

PAPP- 

12 

11 

11 

10 

10 

54 

BETA: 

17 

30 

16 

22 

22 

107 

Total: 

51 

68 

49 

32 

32 

232 

Legcndn  (between  brnckcts:  numbers): 

TO      :  Students  oFUie  Department  oFEducntion  (the  Dutch  name  is  Tocgepnste 

Onderwijskundc:  TO) 
PAPP  :  Students  ofthc  Faculty  oFPublic  Administration  nnd  Public  Policy 
BETA  :  Students  of  the  Departments  of  Computer  Science  (30),  Physics  (5), 

iMnthemntics  (4),  Chemicnl  Engineering  (21),  Electrical  Engineering  (20), 
Mecbnnicnl  Engineering  (15),  nnd  Business  Administration  (12) 
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Subject  Characteristics 

The  following  tests  were  administered,  in  all  cases  selected  because  of  possible 
influence  of  the  pertaining  variable  on  self-chosen  segment  length: 

Group  Embedded  Figures  Test  (GEFT):  The  GEFT  was  used  as  this  test  appears 
to  correlate  to  the  cognitive-style  dimension  "field  dependence"  (Witkin,  Oltman, 
Raskin,  &  Knrp,  1971).  In  the  experiment  a  possible  relationship  between 
segment  length  and  field  dependence  was  assumed,  as  field-independent  subjects 
may  have  more  possibilities  than  field-dependent  subjects  to  see  through  the 
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information  structure  of  the  experimental  video  program  and  tune  their  stepping 
places  accordingly.  Field-dependent  subjects  may,  on  the  other  hand,  have 
problems  with  the  fact  that  in  the  VAR  and  CROSSED  conditions  the  decision 
where  to  stop  has  to  be  taken  by  themselves  without  any  guidance  from  the 
program  (apart  from  the  fact  that  they  know  that  all  presented  factual 
information  has  to  be  stored  in  memory  for  later  recall).  This  may  lead  them  to 
play  safe  by  stopping  frequently,  probably  more  frequently  than  field- 
independent  subjects  might  do. 

CLOZE-test:  Part  of  the  experimental  task  of  the  subjects  was  to  try  to 
remember  as  much  as  possible  of  the  "verbal  information"  that  is  presented  by 
the  video  segments.  This  performance  may  be  influenced  by  verbal  ability. 
Subjects  were  ranked  in  this  respect  by  their  scores  on  a  CLOZE  test.  The 
CLOZE  test  used  consisted  of  two  short  texts  in  which  every  fifth  word  was 
replaced  by  an  empty  space.  The  total  amount  of  deletions  was  91.  Subjects  had 
to  try  to  reconstruct  the  original  texts  by  filling  in  the  right  words. 

STAR-PLUS-test:  This  concerns  a  modification  of  the  familiar  sentence-picture 
comprehension  and  verification  task  of  Chase  and  Clark  (1972),  Pezdek,  Simon, 
Stoeckert,  and  Kiely  (1987)  showed  that  in  their  experiments  good  television 
comprehenders  were  more  likely  to  utilize  an  imagery-based  strategy  for  storing 
information,  This  result  was  the  reason  to  test  the  subjects  in  the  present  study 
with  the  STAR-PLUS-tcst,  to  be  able  to  determine  whether  the  cognitive  trait  it 
measures  influences  preferred  segment  length2. 


Experimental  setting 

Three  adjacent  rooms  were  used,  situated  on  the  first  floor  c(  one  of  the 
university  buildings. 

The  first  room  was  used  for  administration  by  the  experimental  leader  on  duty 
(the  vesearcher  or  student  assistants)  and  as  a  relaxation  room  during  the 
experiments.  Also  the  programr  of  the  project  found  his  workplace  here  and  was 
continuously  available  for  technical  assistance.  The  subjects  were  received  in  this 
room  and  used  the  room  to  take  pauses  while  working  with  the  experimental 
program  (see  the  description  of  the  procedure  below).  Coffee,  tea  and  -  upon 
request  -  hot  chocolate  were  provided.  In  a  corner,  special  arrangements  were 
made  to  run  the  STAR-PLUS  test  that  was  administered  during  the  second 
session.  At  the  back  of  the  room  were  located  the  entrances  of  the  other  two 
rooms  used  in  the  experiment. 

The  second  room  was  used  for  all  the  paper-and-pencil  testing.  This  test  room 
was  equiped  with  four  small  tables  and  four  chairs:  three  for  subjects  and  one  for 
the  experimental  leader.  (No  more  than  three  subjects  could  participate  in  the 
experiment  at  one  time,  as  this  is  the  number  of  interactive  video  sets  that  was 
available  for  the  experiments.) 

The  actual  experiment  took  place  in  the  third  room.  For  that  purpose,  this  room 
was  equipped  with  three  interactive  video  sets  each  containing  an  MS-DOS 
Personal  computer  (Olivetti  M240)  with  a  built-in  video  overlay  board  (Cameron 

2)     A  dotnilod  description  of  this  teat  goes  beyond  the  purpose  of  this  pnper.  A  complete  description 
is  given  by  Verbngen  (in  prepnrntion). 
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MCS),  a  keyboard  and  a  mouse  (Microsoft);  a  color  monitor  (Barco  CM33);  a 
videodisc  player  (Philips  VP406  or  VP410);  and  headphones  (Sennheiser  IID-414). 
Each  set  was  placed  on  a  table  with  a  surface  of  1,2  by  1,2  meters.  The  tables 
were  placed  so  that  the  distance  between  subjects  was  about  2.5  meters  (which 
respects  their  needs  for  privacy  according  to  ergonomic  criteria,  Oborne,  1987, 
pl99),  Screens  between  the  tables  prevented  subjects  to  look  at  the  video  monitor 
of  a  neighbor,  The  subjects  faced  windows  with  dark  blue  but  slightly 
transparent  curtains  that  allowed  some  visual  contact  with  a  tree-rich  parking  lot 
on  the  university  campus.  It  is  believed  that  this  prevented  arousal  of 
claustrophobic  feelings.  Except  for  the  blue  curtains  and  blue  ac^justable  chairs, 
the  dominant  colors  in  the  room  were  shades  of  grey.  The  lighting  level  was 
adjusted  to  balance  the  luminence  of  the  video  monitors.  It  is  believed  that  all 
these  measures  together  created  a  friendly  atmosphere  in  which  subjects  felt  at 
ease, 


Procedure 

The  procedure  was  the  same  in  all  conditions.  The  experiment  took  place  in  two 
sessions,  Session  1  was  prepared  by  switching  on  the  equipment  one-half  hour 
before  the  arrival  of  the  subjects  (to  reach  a  stable  and  qualitatively  good  colour 
picture  on  the  rather  old  video  monitors  used)  and  by  feeding  data  about  the 
expected  subjects  into  the  computer  (name,  subject  number,  experimental 
condition,  age,  sex,  university  department,  year  of  study).  After  loading  these 
data,  the  video  monitor  turned  to  black,  after  which  the  interactive-video  sets 
stood  waiting  for  the  subjects, 

The  subjects  arrived  at  8:45  (if  they  participated  in  a  morning  -ession)  or  at  13:15 
(for  an  afternoon  session).  (They  had  been  reminded  of  the  first  session  with  a 
letter  that  also  emphasized  that  they  were  exp<  cted  to  appear  with  a  clear  mind 
after  a  good  night's  sleep,  This  letter  was  sent  about  one  week  before  the  first 
session,)  The  session  started  in  the  test  room  with  a  few  words  of  welcome  and  a 
short  explanation  of  the  aim  of  the  experiment  and  the  role  of  the  subjects.  After 
this,  a  pretest  was  administered  that  contained  just  one  assignment:  "Write 
down  anything  you  know  about  the  making  of  cheese".  Tl  ;  introductory  words 
and  the  pretest  took  together  about  10  to  12  minutes  (no  one  subject  appeared  to 
be  able  to  describe  anything  substantial  about  the  cheese-making  process). 

Next,  the  subjects  were  invited  to  enter  the  room  with  the  interactive  video  sets, 
They  were  instructed  to  adjust  their  chairs  to  reach  a  comfortable  seating  posture 
'\nd  to  read  a  one-page  instruction  in  a  version  that  suited  the  experimental 
condition  to  which  they  were  assigned.  This  instruction  repeated  some  of  the 
general  remarks  from  the  oral  instroduction  and  explained  how  to  work  with  the 
equipment,  After  having  observed  that  everyone  had  finished  reading,  the 
exper  mental  leader  asked  whether  there  were  questions  and  answered  these  in 
case  there  were,  Next,  the  subjects  were  invited  to  wear  the  headphones  and  the 
experimental  leader  started  the  interactive  video  program  by  typing  a  password. 

The  program  instructed  the  subjects  that  they  were  about  to  watch  an 
introduction  about  chcesemaking  of  about  4,5  minutes,  which  would  play  without 
stopping  after  clicking  the  mouse.  The  subjects  then  watched  the  introduction 
which  automatically  stopped  after  4,5  minutes  on  a  freeze  frame,  Near  the 
bottom  of  the  screen  a  message  was  diplayed  asking  the  student  to  click  the 
mouse  to  continue.    After  this,  the  program  instructed  the  subjects  that  the 
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introduction  would  be  repeated  but  that  this  time  they  had  to  work  in  the  manner 
in  which  they  were  expected  to  work  with  the  main  program  that  followed  the 
introduction. 

The  subjects  then  worked  through  the  introduction  in  one  of  the  ways  described 
before  (dependent  on  the  experimental  condition).  In  the  VAR  and  CROSSED 
conditions,  the  experimental  leader  took  the  opportunity  to  demonstrate  to  the 
subjects  what  was  expected,  by  showing  how  to  control  segment  length  for  a  first 
short  segment  with  a  length  of  three  information  elements.  By  working  through 
the  introduction  the  second  time,  the  subjects  thus  practiced  their  experimental 
task.  At  the  end  of  the  introduction  the  system  ceased  responding  to  the  subjects, 
giving  instead  a  message  to  ask  the  experimental  leader  for  assistance.  The 
experimental  leader  then  had  a  short  individual  conversation  with  each  subject  to 
find  out  whether  he  or  she  understood  the  task,  and  gave  additional  information 
or  reinforcement  if  necessary.  After  this,  the  main  program  was  put  into  position 
by  again  typing  a  password.  At  this  point,  about  20  to  25  minutes  had  passed 
since  the  introduction  had  boen  started  for  the  first  time. 

The  subjects  worked  through  the  main  program,  which  typically  took  about  2.5 
hours,  ranging  from  less  than  2  hours  to  more  than  4.5  hours.  Subjects  in  the 
LIN  condition  often  needed  more  time  than  the  others,  subjects  in  the  SL  and  LS 
conditions  less.  At  the  end  the  program  notified  the  subject  to  ask  the 
experimental  leader  what  to  do  next.  The  subject  then  was  invited  to  return  to 
the  test  room  and  two  posttests  were  administered.  The  first  one  consisted  of 
open  questions  to  measure  recall,  the  second  one  of  multiple  choice  questions  to 
measure  recognition.  To  the  second  one  a  questionnaire  was  attached  that  asked 
the  subjects  about  their  strategy  for  deciding  at  which  segment  lengths  to  stop 
(VAR  and  CROSSED  condition  only),  what  they  thought  of  program-controlled 
fixed  segment  lengths  (SL  and  LS  conditions  only),  and  their  opinion  of  the 
program  (all  conditions)3,  Taking  the  tests  took  about  25  to  30  minutes. 

Last,  the  subjects  were  sent  home  with  three  messages;  (a)  Reappear  in  the 
second  session  with  a  clear  mind  as  you  did  this  first  time;  (b)  do  not  study 
anything  about  cheesemaking  between  this  session  and  the  next  one;  and  (c)  do 
not  tell  your  fellow  students  what  has  happened  here,  because  they  may  be  in  the 
sample  of  subjects  and  every  subject  should  start  the  experiment  without 
preknowledge  of  the  experimental  task. 

Session  2  took  place  three  weeks  after  Session  1  and  was  completely  devoted  to 
testing  subjects.  The  morning  session  of  Session  2  started  at  9:15,  the  afternoon 
session  at  13:45.  For  most  tests,  the  test  room  was  used.  As  the  second  session 
started  later  than  the  first  one  and  was  shorter  (about  two  hours  in  total),  all 
tests  could  be  administered  while  subjects  in  session  1  were  busy  with  the 
interactive  video  program.  The  tests  were  administered  in  the  following  order: 

-  The  Group  Embedded  Figures  Test  (GEFT) 

-  A  retention  test  with  open  questions, 

-  A  retention  test  multiple-choice  questions. 

-  The  STAR-PLUS  test  (in  the  first  room  using  the  special  equipment  that  was 
installed  there  for  this  purpose). 

-  The  CLOZE  test. 


ERIC 


<*)     The  results  or  the  questionnaire  nre  discussed  elsewhere  (Verhngen,  in  preparation). 
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Data  analysis, 

During  their  work  with  the  experimental  video  program,  all  the  actions  of  the 
subjects  were  logged  As  a  result,  the  following  data  were  available: 

1,  All  relevant  time  intervals  (watching  video,  answering  questions,  pauses,  etc); 

2,  Starting  and  stopping  points  of  segments  (and  thus  number  of  segments  and 
number  of  information  elements  per  segment); 

3,  Questions  answered  correctly  the  first  time. 

4,  Questions  answered  correctly  the  second  time  (after  repetition  of  the  video 
fragment  that  followed  negative  feedback  about  incorrect  firjt  answers). 

Next  to  the  data  from  the  program,  the  scores  of  all  tests  were  available.  All 
scores  were  organized  in  such  a  way  that  answers  to  the  questions  of  the  post- 
and  retention  tests  can  be  directly  related  to  the  pertinent  element  numbers  of 
the  main  program. 

Results 


Background  variables 

Of  the  background  variables  "Age",  "Sex",  "University  Department",  and  "Year  of 
Study",  only  Sex  and  Department  appeared  to  correlate  with  the  research  results. 
However,  the  number  of  male  and  female  students  was  not  equal  between  the 
different  departments.  The  majority  of  the  subjects  from  the  Department  of 
Education  were  women,  while  the  technical  departments  were  represented  by 
nearly  only  male  subjects.  A  multivariate  regression  analysis  revealed  that 
differences  between  fields  of  study  (educational  science,  public  administration  & 
public  policy,  technical  engineering)  explain  some  of  the  differences  in 
experimental  results,  while  sex  differences  do  not4.  Sex  differences  are  therefore 
not  further  taken  into  account  in  this  paper  but  the  differences  between  the 
departments  are.  As  can  ht  seen  from  Table  1,  TO  students  did  not  participate  in 
the  SL  and  LS  conditions,  while  subjects  from  the  other  department  are  rather 
equally  distributed  over  all  conditions. 

Self-chosen  segment  length 

Subjects  appeared  to  differ  substantially  with  respect  to  the  mean  of  their  self- 
chosen  segment  lengths.  This  is  displayed  in  Table  2  and  Figure  1. 


)     More  details  can  bo  found  with  Vorlmgon  (in  preparation). 
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Table  2:   Self-Chosen  Segment  Length  and  University 
Department 


Department 

Mean  number  of 
information  elements 
per  segment 

Standard  deviation 

TO: 

7.42 

5.34 

PAPP: 

13.87 

6.67 

BETA: 

17.66 

15.78 

Total: 

12.70 

11.77 

Lege  n  da: 

TO       :  Deportment  of  Educntion  (the  Dutch  name  is  Xocgcpastc  Qndcrwijskunde:  TO) 

PAPP  :  Fnculty  of  Public  Administration  nnd  Public  Policy 

BETA  :  Departments  of  Computer  Science,  Physics,  Mathematics,  Chemical 

Engineering,  Electrical  Engineering,  Mechanicnl  Engineering,  and  Business 

Administration 


NUMBER  OF  SUBJECTS  PER  CLASS 
40  f  


0     5    10    15    20    25    30    35    40    45    50    55    60    65    70    75    80    85    90  95 
CLASS  MEANS  OF  MEAN  SEGMENT  LENGTHS 

VAR  and  CROSSED  conditions  pooled. 

Overall  mean  segment  length:  12.19  information  elements;  standard  deviation:  11.77. 
Minimum:  2.19  information  elements;  maximum:  87.60  information  elements. 

Figure  1:  Distribution  of  means  of  self-chosen  number  of 
information  elements  per  segment 
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The  differences  between  the  mean  segment  hngths  of  the  students  from  the 
different  departments  appears  to  be  significant  (Chi^  =  38.61;  p<.0001).  Because 
of  the  fact  that  no  TO  students  participated  in  the  SL  and  LS  conditions  (Table  1) 
and  the  fact  that  TO  students  appear  to  choose  shorter  segment  lengths  than 
students  from  other  departments  (Table  2),  TO  students  are  only  included  in 
analyses  where  the  VAR,  CROSSED  or  LIN  conditions  are  considered.  On  all 
occasions  where  also  the  SL  and  LS  conditions  are  at  stake,  the  TO-students  are 
thus  left  apart. 

Although  Table  2  shows  that  the  mean  segment  length  of  PAPP  students  and 
Bfeta  students  are  different  too,  it  is  assumed  that  no  reason  exists  to  treat  PAPP 
students  separately  because  in  all  conditions  about  an  equal  number  of  randomly 
assigned  PAPP  students  were  represented. 

Within-program  performance  as  a  function  of  self  chosen  segment  length. 

Figure  2  shows  the  performance  of  subjects  while  working  with  the  program  as  a 
function  of  the  mean  of  self-chosen  segment  lengths. 


PER  CENT  CORRECT 

100  r  


0  5  10  15  20  25  30  35 


MEAN  SEGM.  LENGTH  (INFORMATION  ELEMENTS) 


•    MEAN  SCORE       —  REGRESSION 


VAR  nnd  CROSSED  conditions  pool  jd; 

data  from  all  subjects  with  mean  s'Mf-chosen  segment  length  <  36  information  elements. 

Figure  2:  Within-program  performance  as  a  function  of  mean 
segment  length 

The  figure  shows  for  every  mean  segment  length  the  mean  percentage  of  correct 
answers  to  the  questions  within  the  program  when  they  were  posed  for  the  first 
time,     Only  subjects  with  mean  segment  lengths  shorter  or  equal  to  36 
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information  elements  per  segment  are  included*  The  three  subjects  with  a  longer 
mean  segment  length  (see  Figure  1)  are  considered  as  outlayers5, 

The  regression  line  for  all  subjects  that  describes  the  relation  between  the  mean 
of  self-chosen  segment  length  in  the  'VAR'  and  'CROSSED'  conditions  and  the 
mean  percentage  of  correct  answers  within  the  program  follows  from  the 
equation6*7: 

{MEAN^SCORE)8C,f,cho8cn  ,cnglh  =  85.10  -  .36* {MEAN J3EGMENT_LENGTH J 


The  regression  analysis  explains  only  1,6%  of  the  variance  in  the 
MEAN_SCORE8olf 

chosen  length  variable* 

The  values  on  the  X-axis  of  Figure  2  are  means  of  segment  lengths  that  may  be 
composed  of  contributions  of  a  wide  variety  of  segment  lengths  dependent  on  the 
variance  per  subject.  The  next  figure  (Figure  3)  shows  the  percentage  of  correct 
answers  within  the  program  as  a  function  of  absolute  segment  length. 


The  chosen  criterion  is  that  outlayers  have  mean  segment  lengths  that  are  longer  than  the 
overall  menn  segment  length  plus  two  times  the  standard  deviation.  The  exact  measure  is  35.73 
(which  is  12.19  pirn;  2*11.77) 

6)  In  this  and  the  following  equations,  "SCORE"  means  "PERCENTAGE  OF  CORRECT 
ANSWERS'. 

7)  MNrEANJ5EGMENT_LENC.Tir  means  that  the  values  on  the  x-axis  represent  the  means  of  self- 
chosen  segment  lengths  per  subject. 
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PERCENT  CORRECT 
100  r   


40 


20 


0  5  10  15  20  25  30  35 

ABSOLUTE  SEGM.  LENGTH  (INFO.  ELEMENTS) 

•    MEAN  SCORE   REGRESSION 


VAR  nnd  CROSSED  conditions  pooled; 

only  data  form  subjects  that  occurred  at  least  10  times  arc  used  in  this  analysis. 

Figure  3:  Within-program  performance  as  a  function  of  absolute 
segment  length 


To  arrive  at  this  figure  all  segments  of  one  length  were  pooled,  regardless  of  with 
which  subject  a  segment  was  found.  Only  segment  lengths  for  which  at  least  10 
observations  were  available  were  included  in  the  analysis.  Per  segment  length 
the  mean  was  computed.  The  resulting  values  were  plotted  in  the  figure  with  the 
iiMc'dd  regression  line.  The  resulting  regression  line  for  all  subjects  in  the 
'VAa    CROSSED'  combination  follows  from  the  equation8: 

tMEAN_SCORE}abflolutc  ]ongth  =  85.48 .  .36* {ABSOLUTE JJ EGMENTJJENGTH) 

This  regression  analysis  explains  45.9%  of  the  variance  in  the 
MEAN_SCOREnhaolutc  |ongth  variable. 


The  relation  between  self-chosen  segment  length  and  performance  on  post  tests  and 
retention  tests 

Figure  4  shows  the  relation  between  self-chosen  mean  segment  length  and  test 
performance.  Only  regression  lines  are  displayed.  As  is  clear  from  the  figure, 
subjects  with  longer  self-chosen  segment  lengths  performed  better  than  subjects 
who  chose  shorter  ones.  The  explained  variances  of  the  posttests  and  retention 


8)     Sec  footnote  6. 
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tests  are,  however,  limited  to  18%  (Posttest  1),  12%  (Posttest2),  9%  (Retention 
Test  1),  and  6%  (Retention  Test  2). 


100 


PER  CENT  CORRECT 


programme 


60 


40 


20 


retention  test  2 


retention  test  1 


10  15  20  25 

MEAN  SEGM.  LENGTH  (INFORMATION  ELEMENTS) 


30 


35 


Programme  =  score  on  questions  within  the  program. 

Data  from  all  students  in  the  VAR  and  CROSSED  conditions. 

Figure  4:  Test  Performance  as  a  Function  of  Mean  Segment 
Length 


Differences  in  test  performance  between  all  conditions 

Figures  5,  6,  and  7  and  Tables  3,  4,  and  5  show  for  all  conditions  the  performance 
while  working  with  the  program  and  the  test  results  on  the  posttests  and  the 
retention  tests.  This  is  done  for  the  program  as  whole  as  well  as  for  the  first 
and  second  halves  of  the  program  separately.  This  was  done  to  see  whether  the 
different  treatments  per  half  in  the  VAR,  CROSSED,  SL,  and  LS  conditions 
affected  the  results. 
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Programme  =  score  on  questions  within  the  program.  Data  from  all  students  except  TO, 
Figure  5:  Test  Performance  by  Condition  (Whole  Program) 
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Figure  6:  Test  Performance  by  Condition  (First  Half  of  Program) 
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PERCENT  CORRECT 
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Programme  =  score  on  questions  within  the  prcgram.  Data  from  all  students  except  TO, 

Figure  7i  Test  Performance  by  Condition  (Second  Half  of 
Program) 


Table  3:   Performance  Within  the  Program  Compared  (ail 
conditions) 

PROGRAM 

Conditions  Whole  Part  Part 

1  2 


LIN  <  VAR,  CROSSED,  SL,  LS 
SL  >  VAR,  CROSSED 
SL>LS 
SL  <  LS 

LS  >  VAR,  CROSSED 
LS<VAR 


Legend: 

The  table  shows  difference*  in  pcrfonrnncc  level  between  the  conditions  with  respect  to 
answering  questions  within  the  program  for  the  first  time.  All  questions  were  open 
questions.  Differences  were  evnluntcd  with  a  Chi^  test.  Only  significant  differences  nre 
indicated.  Part  1  means  first  half  of  the  program;  Part  2  means  second  half, 
*:  Level  of  significance  <  .05;  **:  Level  of  significance  <  ,01 


** 
** 
** 


*+ 


** 
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Table  4:  Performance  on  Posttests  Compared  (all  conditions) 


Conditions 

POSTTEST  1 
Whole  Part  Part 
1  2 

POSTTEST  2 
Whole  Part  Part 
1  2 

LIN  >  VAR 

* 

LIN  <  VAR 

** 

LIN  >  CROSSED 

** 

** 

LIN  >  SL 

** 

LIN  <  SL 

* 

VAR  >  CROSSED 

** 

VAR  <  CROSSED 

** 

** 

VAR>LS 

** 

CROSSED  >  SL 

** 

** 

CROSSED  <  SL 

** 

** 

CROSSED  >  LS 

* 

CROSSED  <  LS 

* 

SL>LS 

* 

* 

LS>SL 

** 

Legend: 

The  tnblc  shows  differences  in  performance  level  between  the  conditions  with  respect  to 
post-test  scores.  Po9ttcst  1  contnincd  open  questions  that  referred  to  66  of  the  information 
elements  of  the  program.  Posttest  2  contained  20  multiple  choice  questions,  Differences 
were  evaluated  with  a  Chi^  test.  Only  significant  differences  are  indicated.  Part  1  means 
first  half  of  the  program;  Part  2  means  second  half. 
*:  Level  of  significance  <  .05;       Level  of  significance  <  .01 

Table  5*  Performance  on  Retention  Tests  Compared  (all 
conditions) 

RETENTION  RETENTION 


TEST  1 

TEST  2 

Conditions 

Whole  Part 

Part 

Whole  Part  Part 

1 

2 

1  2 

LIN  >  VAR 

*  * 

LIN  >  CROSSED 

* 

** 

LIN  >  SL 

* 

LIN  >  LS 

* 

**       *  ** 

VAR  >  CROSSED 

*  * 

CROSSED  <  SL 

* 

** 

* 

CROSSED  >  LS 

** 

** 

SL>LS 

** 

** 

The  table  shows  differences  in  performance  level  between  the  conditions  with  respect  to 
retention-test  scores.  Retention  Test  1  contained  open  questions  that  referred  to  81  of  the 
information  elements  of  the  program.  Retention  Test  2  contained  24  multiple  choice 
questions.  Differences  were  evaluated  with  a  Chi*^  test,  Only  significant  differences  arc 
indicated.  Part  1  means  first  half  of  the  program;  part  2  means  second  half. 
*:  Level  of  significance  <  .05;       Level  of  significance  <  .01 
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Mean  segment  length  as  a  function  of  the  position  in  the  program 

Figure  8  shows  the  mean  segment  length  as  a  function  of  the  position  in  the 
program.  Each  information  element  was  part  of  one  segment  per  subject,  This 
means  that  every  information  element  was  part  of  as  many  segments  ns  there 
were  subjects.  In  Figure  8,  the  values  of  Mean  Segment  Length  for  each  Element 
Number  were  computed  per  element  as  the  mean  length  of  all  segments  of  which 
an  element  was  a  part.  This  was  done  for  the  VAR  and  CROSSED  conditions 
separately  To  be  able  to  observe  trends  (for  instance:  Do  subjects  choose  longer 
segments  near  the  end  or  just  the  reverse?),  only  the  data  of  subjects  with  12 
segments  or  more  are  used  in  this  figure.  Figure  8  is  also  used  to  discuss  fatique 
effects  (Research  Question  4), 


MEAN  SEGM.  LENGTH  (INFORMATION  ELEMENTS) 


7^- 


  _  I   I    ...        I    I       ...      I      .....  I.  ..-   1 

34         54         74         94        114        134        154  174 

ELEMENT  NUMBER 


_  !  ..  i    1  .  . 

194       214       234  252 


VAR 

VAR  MINUS  CROSSED 


  CROSSED 

*     MIDDLE  Of  PROGRAMME 


Only  data  of  subjects  with  12  segments  or  more  were  used  in  this  nnalysis, 

Figuic  P:  Mean  Segment  Length  by  Element  Number 


Discussion 

Preferred  .segment  length 

The  first  research  question  asked  which  segment  length  learners  Might  prefer  if 
they  can  choose  segment  length  for  themselves.  With  the  present  group  of 
subjects  the  answer  is  that  self-chosen  segment  length  varies  over  a  wide  range 
(Figure  1).  Apart  from  a  few  extreme  scores,  mean  segments  length  varies  from 
about  2  to  36  information  elements.  Although  segment  length  is  defined  in  terms 
of  semantically  relevant  information  elements,  it  is  illustrative  to  mention  the 
duration  of  the  different  segments  as  they  were  apparently  preferred.  The  main 
program  has  a  length  of  31.5  minutes  and  contains  219  information  elements. 
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This  leads  to  a  mean  length  of  8,63  seconds  of  one  information  element.  The 
mean  of  self-chosen  segment  length  thus  ranges  from  about  18  seconds  to  311 
seconds,  Mere  preference  seems  thus  to  rule  segment  length. 


Segment  length  and  direct  recall 

The  second  research  question  was  whether  a  relation  exists  between  segment 
length  and  direct  recall  and  if  it  makes  a  difference  whether  segment  length  is 
self-chosen  or  program  controlled.  Self-chosen  segment  length  is  discussed  first 

Figure  2  shows  that  the  mean  cf  self-chosen  segment  length  has  only  a  weak 
relationship  with  direct  recall,  According  to  the  regression  line  in  Figure  2 
subjects  with  a  mean  segment  length  of  2  information  elements  answered  about 
84%  of  within-program  questions  correctly  the  first  time,  while  subjects  with  a 
mean  segment  length  of  36  information  elements  answered  about  72%  of  these 
questions  correctly.  This  is  a  relatively  small  difference  given  the  difference  in 
mean  segment  length, 

It  is,  however,  true  that  the  regression  line  from  Figure  2  explains  only  2,5%  of 
the  variance  of  the  performance  on  within-p*.  ogramme  questions.  It  is  therefore 
interesting  to  observe  that  in  the  given  situation  Figure  3  suggests  that  the  mean 
of  all  scores  per  absolute  segment  length  is  a  reasonable  predictor  of  learner 
performance  for  that  segment  length  (with  an  explained  variance  of  almost  46%). 
The  similarity  between  the  lines  of  Figures  2  and  3  is  thereby  striking.  The  small 
slope  of  the  regression  lines  reinforces  the  idea  that  subjects  know  for  themselves 
how  much  information  they  can  handle  in  one  time  and  decide  what  segment 
length  to  choose  accordingly, 

The  greater  precision  of  absolute  segment  length  may  be  caused  by  the  fact  that 
subjects  make  conscious  decisions  about  segment  length  on  a  segment-by- 
segment  basis,  This  may  cause  much  variance  in  the  segments  lengths  of 
individual  subjects  (of  which  Figures  1  and  2  are  derived).  It  may,  however, 
cause  limited  variance  in  performance  per  segment  length  if  each  subject 
optimally  tunes  his  or  her  attention,  ability,  and  mental  effort  to  each  segment 
that  is  presented  and  chooses  the  length  of  that  segment  accordingly,  It  has  to  be 
noted  that  subjects  with  better  memories  have  thereby  obviously  more 
possibilities  to  decide  what  segment  length  they  prefer,  The  data  suggest  that  in 
this  respect  technical  students  have  the  most  room  to  manoeuvre  (see  Table  2), 

The  question  whether  the  within-program  performance  of  subjects  in  the  variable 
conditions  differs  from  the  performance  in  the  conditions  with  fixed  segment 
length  can  be  discussed  on  the  basis  of  Figures  5,  6,  and  7  and  Table  3.  The  VAR 
and  CROSSED  conditions  show  no  significant  differences  in  this  respect,  The 
subjects  in  the  LIN  condition  appear  however  to  perform  worse  than  all  other 
conditions  (around  55%  while  all  other  conditions  are  doing  around  75%  to  80%). 
Answering  213  questions  in  one  time  about  31,5  minutes  of  video  obviously  is  a 
difficult  job, 

If  the  first  and  second  half  of  the  program  are  considered  separately,  the  SL  and 
LS  conditions  appear  to  differ  from  the  VAR  and  CROSSED  conditions  and  from 
each  other  (Figures  6  and  7,  and  Table  3).  SL  subjects  did  better  during  the  first 
half  of  the  program  than  subjects  in  all  other  conditions,  while  LS  subjects  did 
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better  during  the  second  half.  It  is  clear  that  the  SL  and  LS  subjects  experienced 
the  short  segments  as  relatively  easy.  The  long  segments  were  more  difficult  for 
th  m.  LS  subjects  did  significantly  worse  than  the  SL  subjects  during  the  first 
half  of  the  program  and  SL  subjects  did  significantly  worse  than  the  LS  subject? 
during  the  second  half.  From  all  other  differences,  only  the  different  performance 
of  LS  subjects  and  VAR  subjects  during  the  first  half  of  the  program  was 
significant;  the  LS  subjects  did  worse  than  the  VAR  subjects. 

The  order  of  magnitude  of  the  significant  differences  between  the  variable 
conditions  and  the  SL  and  LS  conditions  is  not  impressive.  The  differences  are 
similar  to  the  differences  within  the  variable  conditions  according  to  Figures  2 
and  3.  The  relation  between  segment  length  and  performance  level  is  obviously 
rather  weak,  regardless  of  the  fact  whether  segment  length  was  self-chosen  or 
program  controlled  (under  the  condition  that  program-controlled  segment  length 
falls  within  the  range  of  self-chosen  segment  lengths). 


Segment  length  and  delayed  recall 

The  third  research  question  asked  whether  a  relation  exists  between  segment 
length  and  delayed  recall  and  if  it  makes  a  difference  whether  segment  length  is 
self-chosen  or  program  controlled. 

Figure  4  shows  that  subjects  with  longer  mean  self-chosen  segment  length 
performed  better  on  the  post  tests  and  retention  tests  than  subjects  who  chose 
shorter  segments.  This  may  be  caused  because  these  subjects  have  better 
memories  or  because  they  just  preferred  longer  segments  and  invested  more 
mental  effort  to  make  that  choice  succesful  (not  to  miss  too  many  questions 
within  the  program),  which  resulted  in  deeper  processing  that  perhaps  paid  itself 
back  on  the  different  post-  and  retention  tests.  A  combination  of  these  two  causes 
is  likely. 

From  Figures  5,  6,  and  7  and  from  Tables  4  and  5  it  is  apparent  that  the  LIN 
condition  did  relatively  well  as  far  as  the  posttests  and  the  retention  tests  are 
concerned.  One  possible  explanation  for  this  could  be  that  this  is  a  consequence 
of  the  relatively  bad  performance  within  the  program,  which  yielded  extra 
exerciso  due  to  repetitions  of  video  fragments.  This  appears,  however,  not  to  be 
true.  An  extra  analysis  showed  that  repetition  of  video  was  negatively  correlated 
with  posttest  and  retention-i  st  performance  in  all  conditions  (more  details  can 
be  found  in  Verhagen,  in  preparation).  It  appeared  that  answering  the  questions 
within  the  program  correctly  at  the  first  attempt  was  a  good  predictor  of  correctly 
answering  questions  about  ihe  pertinent  information  elements  in  later  tests* 

An  alternative  explanation  of  the  good  performance  of  LIN  subjects  might  be  that 
the  LIN  subjects  invested  more  mental  effort,  as  they  knew  that  they  would  be 
questioned  about  the  complete  program.  Similar  to  the  assumption  about  the 
better  performance  of  subjects  with  longer  mean  segments  in  the  variable 
conditions  (Figure  4),  here  also  this  extra  mental  effort  could  have  caused  better 
remembering.  This  idea  is  reinforced  by  the  fact  that  the  SL  and  LS  conditions 
yielded  relatively  low  posttest  scores  for  the  program  halves  in  which  the  vvithin- 
program  task  was  easy  (where  subjects  worked  with  short  segments). 

The  effect  of  mental  effort  has  been  discussed  by  Salomon  (1984).  According  to 
Salomon,  the  Amount  of  Invested  Mental  Effort  (AIME)  is  related  to  the 
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Perceived  Demand  Characteristics  (PDC)  of  a  task,  and  the  Perceived  Self 
Efficacy  (PSE)  for  that  task.  PDC,  PSE  and  AIME  can  be  balanced  in  many 
ways,  yielding  different  levels  of  learning.  The  performance  level  while 
answering  questions  that  a  subject  considers  to  be  acceptable  thereby  forms  one 
factor  that  can  be  balanced  with  preferred  segment  length  in  the  variable 
conditions.  The  fact  that  mean  segment  length  differs  considerably  in  these 
conditions  means  that  different  subjects  made  different  trade-offs  with  respect  to 
their  memory  capacity  and  their  willingness  to  invest  mental  effort.  It  may 
however  be  that  in  general  subjects  in  the  variable  conditions  preferred  a 
relatively  relaxed  approach  to  their  task  and  thus  choosed  segment  lengths  that 
allowed  reasonable  performance  levels  with  a  moderate  AIME.  This  could 
explain  why  LIN  subjects,  who  faced  harder  PDCs,  performed  relatively  well  on 
the  different  post-  and  retention  tests.  They  tuned  their  AIME  to  the  task  and  as 
the  task  was  difficult,  their  AIME  was  substantial  which  caused  relatively  good 
storage  in  memory. 

Similar  reasoning  can  be  applied  to  the  SL  and  LS  conditions.  In  these  conditions 
the  posttest  questions  ajd  the  retention-test  questions  were  answered  relatively 
well  for  those  halveo  of  the  program  where  the  subjects  worked  with  long 
segments  (which  was  relatively  difficult)  and  relatively  badly  for  the  halves 
where  they  worked  with  short  segments  (and  felt  no  need  to  invest  much  mental 
effort). 

In  all,  the  results  suggest  that  designers  have  much  freedom  in  choosing  segment 
lengths  when  developing  interactive  video  programs,  and  also  if  they  decide  to 
use  segments  with  fixed  lengths.  The  point  is  to  inform  learners  of  the  PDC  of 
each  segment  ahead  and  motivate  them  to  tune  their  AIME  to  the  needed  level. 
For  this,  in  principle,  many  audiovisual  communication  options  are  available  (see 
again  Verhagen,  in  preparation,  for  a  more  detailed  discussion  of  this  issue). 

A  last  observation  with  respect  to  delayed  recall  is  that  there  appear  to  be 
recency  effects.  This  follows  from  the  differences  between  the  scores  on  the  two 
halves  of  the  program  of  the  VAR  and  CROSSED  conditions  in  particular, 
especially  where  the  posttests  are  concerned.  As  is  apparent  from  Figures  6 
and  7  and  from  Table  4,  the  VAR  subjects  sccred  better  than  CROSSED  subjects 
on  posttest  questions  with  res  :>ect  to  the  second  half  of  the  program  and  worse  on 
questions  about  the  first  half,  while  the  scores  on  the  posttests  about  the  program 
as  a  whole  (Figure  5)  showed  no  noticable  differences.  VAR  subjects  saw  the 
second  half  of  the  program  later  than  the  first  half  and  the  CROSSED  subjects 
just  the  other  way  around.  The  mean-time  difference  between  the  two  halves  was 
about  one  hour  and  15  minutes.  The  most  probable  explanation  for  the  different 
test  results  is  therefore  that  forgetting  starts  immediately  after  learning  and  that 
after  75  minutes  the  forgetting  curve  is  already  important  enough  to  cause  the 
observed  differences. 


Information  load  as  a  function  of  the  position  in  the  progam 

Under  the  assumption  that  the  influence  of  individual  differences  of  subjects  will 
disappear  if  the  mean  segment  length  for  each  information  element  in  Figure  8 
are  obtained  as  the  mean  of  many  subjects,  the  "ideal"  curve  would  be  a  straight 
horizontal  line.  In  that  case  form  and  content  of  the  program  would  have  no 
influence  on  preferred  segment  length  in  the  sense  that  the  information  load  of 
the  program  would  then  be  perceived  to  be  constant  through  cut. 
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This  appears  not  to  be  the  case.  This  is  reason  to  analyse  more  carefully  various 
formal  features  of  the  program  such  as  factors  like  complexity  of  narration, 
mutual  influence  of  picture  and  sound,  use  of  super-imposed  texts,  and 
information  load  due  to  technical  terms.  The  analysis  of  these  factors  goes 
beyond  the  scope  of  this  paper.  More  information  can  be  found  with  Verhagen  (in 
preparation),  A  notable  aspect  is  the  occurrence  of  the  peak  before  the  middle  of 
the  program.  It  concerns  the  only  part  of  the  program  with  an  animation 
sequence.  This  sequence  has  however  also  some  repetition  of  content  elements  in 
it.  The  conclusion  that  the  animation  sequence  is  perceived  as  more  easy  as  the 
normal  video  sequences  can  thus  not  be  drawn  without  further  analysis. 

A  clear  trend  that  shows  whether  subjects  decided  to  take  longer  segments 
towards  the  end  of  the  program  or  on  the  contrary  chosed  shorter  ones  did  not 
emerge. 

Fatigue  effects 

The  last  research  question  was  whether  fatique  effects  affect  self-chosen  segment 
length.  Figure  8  showes  that  there  appears  to  be  no  reason  to  take  these  into 
account.  The  curves  of  the  VAR  and  CROSSED  conditions  are  very  similar  with 
the  exception  of  the  dip  in  the  curve  of  the  CROSSED  condition  round  the  middle 
of  the  program.  There  is,  however,  an  easy  explanation  for  this.  The  middle  of 
the  program  was  the  forced  end  of  the  program  for  the  CROSSED  subjects,  They 
thus  could  not  choose  the  length  of  their  last  segment  freely.  This  last  segment 
appeared  often  to  be  relatively  short,  which  caused  the  mean  segment  length  to 
drop  on  that  place  (an  effect  that  can  also  be  observed  at  the  end  of  the  program 
for  both  conditions,  where  the  VAR  subjects  stopped  and  the  CROSSED  subjects 
were  half  way).  Similarly,  in  the  CROSSED  condition  elements  just  after  the 
middle  could  not  be  part  of  segments  that  began  before  the  middle,  causing  these 
elements  to  be  part  of  relatively  many  short  segments.  Apart  from  this,  segment 
length  appears  not  to  be  substantially  different  between  the  VAR  and  CROSSED 
conditions.  Given  the  mean  time  difference  of  75  minutes  with  which  VAR  or 
CROSSED  conditions  arrived  at  a  similar  point  in  the  program,  it  seems  that 
fatigue  effects  did  not  affect  segment  length. 


Conclusion 

The  main  results  of  the  study  are: 

1.  Self-chosen  mean  segment  length  varies  from  less  than  two  information 
elements  up  to  more  than  36  information  elements,  with  only  a  small  increase 
in  the  amount  of  wrong  answers  to  questions  within  the  program.  Subjects 
seem  to  acftust  the  amount  of  information  they  choose  to  get  in  one  time  to 
their  memory  capacities  and  to  their  willingness  to  invest  mental  effort. 

2.  If  a  difficult  task  is  expected,  subjects  seem  to  adapt  themselves  by  investing 
more  effort.  This  could  be  the  reason  why  the  subjects  in  the  LIN  condition 
performed  relatively  well  on  posttests  and  retention  tests  and  why  subjects  in 
the  SL  and  LS  conditions  did  better  in  this  respect  .vhen  questions  related  to 
that  half  of  the  program  in  which  they  worked  with  long  segments, 

3.  No  differences  are  observed  between  the  VAR  and  the  CROSSED  conditions 
which  can  be  attributed  to  fatique  effects.  Posttest  scores  show  a  recency 
effect,  suggesting  that  the  time  difference  between  VAR  and  CROSSED  of 
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about  75  minutes  yields  so  much  forgetting  about  the  program  half  that  was 
studied  first,  that  this  already  affects  posttest  performance. 

The  wide  variety  of  preferred  length  yields  little  guidance  for  designers  who 
design  interactive  video  programs  with  fixed-length  segments.  The  main 
message  may  be  that  quality  communication  can  be  accomplished  in  many  forms, 
whereby  different  audiovisual  formats  and  segment  lengths  may  appear  feasible 
as  long  as  learners  start  their  tasks  with  a  realistic  expectancy  of  the  demand 
characteristics  and  can  be  motivated  to  tune  their  mental  effort  accordingly.  A 
further  discussion  goes  beyond  the  purpose  of  this  paper.  Detailed  discussion  and 
further  analyses  are  reported  elsewhere  (Verhagen,  in  preparation). 
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BrtCit£r0Mm/--Constructivists  argue  that  knowledge  is  constructed  only  in  the  mind  of  the 
learner,  and  that  good  teaching  is  not  necessarily  related  to  good  learning.  Bodner  (1986)  states 
that  "until  recently,  the  accepted  model  for  instruction  was  based  on  the  hidden  assumption  that 
knowledge  can  be  transferred  intact  from  the  mind  of  the  teacher  to  the  mind  of  the  learner."  He 
goes  on  to  claim  that  "teaching  and  learning  are  not  synonymous;  we  can  teach,  and  teach  well, 
without  having  the  students  learn." 

The  theory  holds  (Von  Glasersfeld,  1979)  that  constructivism  is  facilitated  by  having  the 
learner  identify  topics  or  issues,  locate  resources,  plan  investigations  and  activities,  practice 
self-evaluation,  and  formulate  principles.    The  emphasis  is  shifted  from  these  activities  as 
those  that  teachers  do,  to  those  that  students  should  perform. 

In  contrast  to  widespread  emphasis  on  instructional  technology  as  "tools  for  teaching"  the 
emphasis  in  this  study  was  on  using  technology  as  a  vehicle  for  learning.  Hypermedia  was  used 
to  foster  constructivism  in  science  and  mathematics  in  K-12  settings.   Goals  of  the  study 
included: 

•  Students  and  teachers  would  gain  skill  in  using  computer- 

based  technology 

•  Students  would  increase  their  understanding  of  math  and 

science 


Technology  was  used  as  a  hook  to  entice  students  to  learn  math  and  science  and  to  provide  an 
alternative  to  "read-a-book,"  "write-a-paper,"  and  "take-a-test."    It  was  hypothesized  that 
students  might  more  readily  use  primary  information  sources  and  texts  if  the  goal  were  to 
create  a  hypermedia  program.  The  reason  for  emphasizing  student  production  was  that,  to 
foster  constructivism,  students  need  to  display  what  they  learn.  As  one  student  said,  "You  really 
have  to  know  trigonometry  if  you're  going  to  develop  a  program  about  it." 

In  every  case  student  producers  created  the  program  architecture  and  completed  the  content 
treatment  for  certain  portions  of  the  program.   The  teacher  assisted  by  providing  resources  and 
clarifying  questions  of  content.  With  one  or  two  elements  worked  out  in  detail,  remaining 
portions  of  each  program  were  left  in  skeleton  form  so  that  other  students  in  the  class  could 
complete  them.  The  process  of  completing  an  element  included  using  scanned  diagrams,  digitized 
sound,  written  material,  and  selected  footage  from  videodiscs,  based  on  the  nature  of  the  math  or 
science  concepts  being  portrayed. 

Method- A  consortium  was  formed  of  K-12  schools  in  central  Iowa,  their  Area  Education 
Agency,  and  the  College  of  Education  at  Iowa  State  University.    Twelve  schools  were  chosen  to 
participate  in  the  project  entitled  EMPOWERING  STUDENTS  WITH  HYPERMEDIA,  extending  from 
September  1990  to  April  1991. 

Four  in-service  sessions  were  held  for  teacher/student  production  teams  throughout  the  year  to 
help  them  develop  scenarios  for  their  programs,  and  to  learn  how  to  use  HyperCard,  Linkway, 
and  HyperStudio. 


9 

ERJC 


670  9».  5 


Teachers  served  primarily  as  content  advisors;  students  became  program  designers.  University 
personnel  and  Area  Education  Agency  collaborators  guided  teachers  and  students,  conducted  in- 
service  sessions,  and  provided  technical  advice.  It  was  particularly  important  to  teachers  that 
they  choose  students  who  had  an  interest  in  the  content  and  in  the  technology,  and  who  were  more 
or  less  self  starters. 

Typical  projects  included  Tells,  Tissues,  Organs,  Organ  Systems," 
"Examining  the  Sphere,  "  and  "Reactions  of  Acids  and  Bases."  The  computer  programs  were 
created  to  be  open-ended,  in  that  other  students  in  the  class  could  add  their  own  information  as 
they  studied  the  concepts.  For  example,  student  producers  might  develop  a  "lesson"  on  the 
heart,  and  let  it  serve  as  a  model  for  other  students  who  might  develop  a  lesson  on  the  liver  or 
the  brain. 

Classroom  teachers  served  as  content  specialists  and  facilitators  for  their  student  production 
teams,  but  they  did  relatively  little  programming.  The  main  role  of  'he  teachers  was  to  guide 
students  through  the  content  of  mathematics  and  science  and  to  suggest  resources  that  students 
could  incorporate  into  the  projects. 

University  advisors  and  Area  Education  Agency  personnel  conducted  the  in-service  programs, 
advised  on  technical  questions,  and  assisted  in  purchasing  equipment  to  support  the  projects. 

Field  r»'/— Finished  programs  were  used  by  target  students  during  the  period  March  through 
May  1991.   Data  were  collected  from  those  students  during  that  time,  as  well  as  from  the 
student  producers  and  their  teachers  (see  instruments  below).  The  response  from  the  12 
cooperating  schools  was  not  large,  probably  for  two  reasons:  (1)  the  specific  subject  of  the 
computer-based  materials  may  have  been  covered  previously  in  the  year,  and  (2)  the  time 
period  for  gathering  data  coincided  with  the  pressure  of  other  activities  at  the  end  of  the 
academic  year.   Thus,  information  reported  here  is  anecdotal  as  well  as  quantitative. 

A  total  of  35  student  users,  3  teachers,  and  3  student  producers  responded  to  the  formal 
instruments. 

* 

Student  users 

The  following  4  charts  depict  student  attitudes  toward  technology,  their  fear  of  it,  their  level  of 
knowledge  before  using  the  materials,  and  their  preference  for  working  on  their  own. 
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Selected  responses  from  student  users'  indicate  the  degree  of  constructivism  they  may  have 
achieved.  When  asked  to  compare  traditional  instruction  (hooks,  TV,  field  trips,  etc.)  to  this 
experience,  students  indicated  it  was  more  interesting,  they  liked  it  better,  they  felt  it  was 
newer  and  faster,  and  they  could  work  at  their  own  pace.  "You  were  the  one  asking  the  questions 
and  getting  the  answers,"  one  student  said  Some  students  wanted  "...more  pictures  and  more 
activities."  When  asked  if  they  would  recommend  the  program  to  a  friend,  -irtually  everyone 
said  yes. 

No  rigorous  determination  was  made  to  determine  how  much  content,  in  this  case  physical 
science,  was  learned,  but  students  did  indicate  that  they  observed  how  chemicals  reacted 
together,  and  that  reactions  were  different  when  acids,  water,  and  bases  were  used.  (The 
program  used  was  based  on  a  videodisc  from  the  American  Chemical  Society,  showing  chemical 
reactions). 

Student  Producers 

Approximately  50  students  across  the  12  projects  served  asprogram  designers  and 
programmers.  Comments  gleaned  from  informal  contact  with  them  during  'iie  workshops,  from 
observations  their  teachers  made,  and  from  the  formal  instruments  they  submitted  indicate  that 
they  did  most  of  the  work  themselves,  under  the  supervision  of  the  teacher.  They  felt  they 
learned  more  about  the  technology  than  about  the  content  area,  and  that  they  enjoyed  working 
alone  or  as  part  of  a  small  design  team.  They  acknowledged  the  great  deal  of  time  needed  to 
produce  the  program  shell,  but--as  one  student  said,  "...it  made  me  manage  my  time."  They 
indicated  that  the  technology-based  activities  helped  the  content  come  alive  for  them,  and  that 
they  acquired  a  working  understanding  of  high  technology  too. 


Teachers  expressed  enthusiasm  for  the  motivational  aspects  of  this  approach,  claiming  that 
some  students  who  had  not  shown  much  interest  in  either  technology  or  math/science  now  were 
much  more  involved  in  both.  They  found  that  their  role,  as  teachers,  was  more  suited  to  posing 
questions,  helping  students  find  resources,  and  monitoring  progress.   Nearly  all  teachers  * 
recognized  the  shift  from  traditional  teacher-centered  instruction  to  student-based  learning. 

Conclusions  and  Future  Work 

The  project  was  considered  successful  enough  that  a  second  phase  is  underway  for  the  1991- 
1992  school  year.    The  former  participants  (both  teachers  and  student  producers)  are  serving 
as  mentors  to  8  new  schools. 

An  organizational  meeting  brought  mentors  and  new  participants  together,  where  they  spent 
time  planning  new  work  with  each  other. 

Some  student  producers  from  the  first  year  have  been  identified  in  their  school  systems  as 
"experts"  who  are  designated  to  work  with  both  faculty  and  students  in  the  use  of  instructional 
technology.   Several  teachers  have  contracted  on  their   wn  to  work  with  neighboring  schools  to 
foster  the  use  of  technology  by  providing  inservice  workshops. 

It  appears  that  the  locus  of  control  for  education  can  be  shifted  to  students,  and  that  technology 
provides  a  mechanism  for  that  to  happen. 


Bodner,  George  M.   Constructivism:  a  theory  of  knowledge.   Jl.  of  Cheni.  Ed.  Vol.  63,  No.  10.   Oct.  1986 

Von  Glasersfeld,  Ernst.    The  concepts  of  adaptation  and  viability  in  a  radical  constructivist  theory  of 
knowledge.   In  New  directions  in  Pjagetim  Theory.  Irving  Sigel,  Ed.   Lawrence  Erlbaum  Associates, 
Hillsdale,  New  Jersey.  1981. 
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Frase  (1970),  Rothkoph  (1970),  Bibiscos  (1967),  Anderson  (1970, 1975)  and  others 
have  produced  a  large  volume  of  research  that  looks  at  the  effect  of  adding  questions 
(adjunct  questions)  to  text  materials  to  improve  learning.  In  general  their  findings  reflect 
that  prequestions  may  serve  an  orienting  function  if  they  are  placed  at  the  beginning  of  a 
text  passage,  facilitating  selective  attention.  The  researchers  reach  this  conclusion  because 
the  subjects  in  the  research  tend  to  remember  information  related  to  the  questions  than 
other  "incidental"  information  in  the  text  However,  there  is  a  fair  amount  of  research 
that  finds  no  facilitating  effect  for  inserted  prequestions.  One  reason,  perhaps,  is  that  the 
learner  does  not  remember  the  prequestions  long  enough  for  them  to  be  of  any  use  in  the  text. 
Another  possible  reason  is  that  the  questions  cue  the  learner  as  to  what  the  text  is  about  but 
they  are  just  not  interested  enough  in  the  content  to  devote  the  effort  needed  to  learning  it. 

Likewise  the  effect  of  questions  following  the  text  and  questions  inserted  in  the  text 
have  been  studied  with  contradictory  results.  In  general  it  has  been  found  that  questions 
placed  after  text  materials  facilitates  long-term  recall  of  information  about  the  text. 
Subjects  given  prequestions  seem  to  learn  more  "incidental"  material  than  those  given 
postquestions,  and  they  seem  to  do  better  on  inferential  questions  as  opposed  to  detail 
questions.  The  researchers  hypothesize  that  post-questions  serve  a  rehearsal  function  that 
facilitates  recall. 


Research  on  feedback  has  looked  at  many  different  independent  variables 
including  type  of  feedback,  timing  of  feedback,  placement  of  feedback,  and  the  learner's 
degree  of  confidence  in  responding.  Dependent  variables  include  the  number  of  questions 
answered  correctly  on  a  post-test  that  were  missed  in  the  material,  the  number  of  correct  in 
the  material  missed  on  the  pcst-test,  and  the  response  patterns  correlated  with  response 
certainty.  Most  of  the  research  in  this  area  is  limited  to  verbal  learning  tasks,  and  the  few 
related  to  concept  learning  tasks  show  no  significant  differences.  The  findings,  in  general 
showfeedback  to  be  superior  to  no  feedback,  but  show  very  few  differences  that  can  be 
attributed  to  the  type  of  feedback.  Kulhavy  (1977)  reports  finding  what  he  calls  a  delayed 
retention  effect,  which  appears  if  the  feedback  is  massed  at  the  end  of  the  instruction 
rather  than  given  immediately  after  the  response.  He  hypothesizes  that  immediate 
feedback  activates  an  interference  effect  caused  by  a  discrepancy  between  the  student's 
answer  and  the  correct  answer.  This  interference  is  reduced  or  eliminated  through  the  use  of 
delayed  feedback,  however,  it  is  moderated  by  other  variables  like  response  certainty. 

How  is  the  research  on  questions  and  the  research  on  feedback  related?  There  is 
virtually  no  reference  of  one  set  of  literature  in  the  other  yet  both  may  looking  at  similar 
phenomenon.  We  believe  there  is  a  connection  between  question  research  and  feedback 
research  findings  that  can  be  viewed  from  an  information  processing  perspective. 

Atkinson  and  Shiffrin  (1968) ,  GagnS  (1985) ,  and  Kumar  (1971)  have  identified 
"stages'1  of  information  processing.  Research  on  learning  has  typically  investigated  the 
effects  of  instructional  intervention  on  one  or  more  of  these  stages.  For  example,  research  on 
adjunct  questions  has  focused  on  the  stage  Kumar  refers  to  as  selective  attention,  whereas 
pre-instructional  strategies  such  as  providing  learning  objectives  seems  to  focus  on  stimulus 
perception  through  the  arousal  of  expectations.  Considering  that  questions  may  serve 
different  roles  in  instruction,  depending  upon  the  stage  of  information  processing  they 
impact  it  is  reasonable  to  expect  that  feedback  might  serve  different  purposes,  as  wel1 

1 


**  This  paper  is  abstracted  from  a  chapter  in  Interactive  Instruction  and  Feedback,  edited 
by  /.  V.  Dempsey  and  G.  Sales,  Educational  Technology  Publications,  in  press. 
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It  is  helpful  to  identify  a  model  of  information  processing  to  use  in  the  organization 
of  question  and  feedback  effects.  The  model  we  use  is  taken  from  Gagn6  (1985)  which  is 
based  on  Atkinson  and  Shiffrin's  (1968)  model  This  model  postulates  features  such  as 
sensory  registers,  short-term  memory,  long-term  memory  and  an  executive  control 
mechanism* 


Information  processing  is  set  into  effect  when  something  from  the  learner's 
environment  stimulates  the  learner  and  activates  receptors  which  transmit  information  to 
the  central  nervous  system.  After  a  brief  registration  in  one  of  the  sensory  registers,  it  is 
transformed  into  recognizable  patterns  put  into  short-terjn  memory  (STM).  This 
transformation  is  referred  to  as  selective  perception.  Storage  in  the  STM  has  a  relatively 
brief  duration,  usually  less  than  20  seconds,  unless  some  sort  of  rehearsal  takes  place. 
Additionally,  STM  is  limited  in  its  capacity:  only  a  few  separate  items  can  be  kept  in  STM 
at  one  time.  In  order  that  information  can  be  remembered  for  longer  periods  and  in  larger 
quantities,  it  must  be  semanticallv  encoded  to  a  form  that  will  go  into  long-term  memory 
(LTM).  Both  information  from  short-term  and  long-term  memory  may  be  passed  to  a 
response  generator  and  is  transformed  into  some  sort  of  action.  The  message  serves  *o 
activate  effectors  which  result  in  a  performance  that  can  be  observed  to  occur  in  the 
learner's  environment 


Although  these  processes  are  internal  to  the  learner,  there  are  external  events  - 
events  outside  the  learner  -  that  can  be  made  to  influence  the  processes  of  learning.  The 
nine  events  include: 

1.  Gaining  attention, 

2.  Informing  learner  of  the  objective, 

3.  Stimulating  recall  of  prerequisite  learning, 

4.  Presenting  the  stimulus  material, 

5.  Providing  learning  guidance, 

6.  Eliciting  the  performance, 

7.  Providing  feedback  about  performance  correctness, 

8.  Assessing  the  performance,  and 

9.  Enhancing  retention  and  transfer. 

It  is  to  these  nine  events  of  instruction  that  we  will  address  the  issue  of  questions  and  any 
ensuing  feedback.  It  is  our  contention  that  questions  and  feedback  may  serve  different 
functions  according  to  which  event  of  instruction  -  that  is,  to  which  internal  process  -  they 
are  being  used  to  enhance. 

Event  1 :  Gain  Attention.  Attention  usually  involves  the  two  processes  of  (1) 
orientation  of  the  receptors  toward  the  stimulus  and  (2)  the  encoding  of  the  stimulus 
(Anderson,  1970),  only  the  first  of  these  two  processes  relates  to  this  first  stage  of 
information-processing.  Although  the  process  of  gaining  attention  is  thought  of  as  an 
initial  event,  control  of  attention  within  the  lesson  has  been  found  to  be  important  when 
students  are  bored,  tired,  or  under  pressure  to  work  quickly,  as  well  (cited  in  Anderson, 
1970).  Questions  presented  as  a  stimulus  for  attention  arousal  will  not  guarantee  that  what 
is  to  follow  will  be  learned.  Unless  this  attention  arousal  does  occur,  however,  there  will 
not  be  adequate  reception  of  the  stimuli  by  the  receptors  to  allow  for  the  other  processes  to 
occur.  Feedback  for  attention  questions  is  probably  rhetorical,  if  given  at  all.  It  may  be 
desirable  to  allow  students  to  express  answers  without  confirmation  of  correctness  if  the 
object  is  to  create  cognitive  dissonance  or  information  search,  for  example,  starting  a  class  by 
asking,  "Why  are  more  men  bald  than  women?"  Attention  questions  might  increa  e 
uncertainty  about  what  the  learner  already  knows,  creating  epistemic  curiosity.  In  this 
case,  there  is  no  need  for  feedback  because  the  purpose  of  the  event  is  to  have  the  student 
seek  the  answer  to  the  question  in  the  subsequent  instruction.  Table  1  below  shows  how 
other  events  might  be  treated  with  regard  to  feedback. 
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Table  1. 

Recommendations  for  the  use  of  questions  and  feedback  related  to  stages  of  information 

processing 


Function  of  the  question 

Function  and  type  of  feedback 

1.  Gaining  Attention 

1.  Arousal,  Create  Cognitive  dissonance. 
Open  ended  questions  with  no  feedback. 

2.  Inform  the  learner  of  the  objective. 

2.  Create  an  expectancy  for  performance. 
Rhetorical  questions  and  didactic  answers 
could  be  used  to  inform  the  student  of  the 
objectives. 

3.  Stimulate  the  recall  of  prerequisite 
learning 

3.  Bring  related  knowledge  into  short-term 
memory  and  confirm  present  knowledge, 
questions  eliciting  analogies  witn  ngnt~ 
wrong  or  conditional  feedback  could  be 
used  to  test  the  understanding  of 
prerequisites.  Pretests  may  serve  the 
same  iunLuun  vvviin  or  wunuui  iwuuaCK/. 

4.  Present  the  stimulus  material 

4.  Socratic  dialog  to  have  the  student  deduce 
what  information  is  needed.  Feedback 
takes  the  form  of  confirmation  of 
responses,  and  probing  questions  to  guide 

in  fin  in/ 

5.  Provide  learning  guidance 

5.  Questions  provide  for  modeling  component 
parts  of  the  skill  being  learned.  Feedback 
should  show  correct  answers  for  analysis 
bv  the  student. 

6.  Elicit  the  performance 

6.  Questions  recall  learned  skill  or 

components  of  the  learned  skill  to  test  for 

iiusunuerbuinuiiig.  rcevii/aLK  snuuiu  v%s 

remedial,  directed  at  the 
misunderstanding  if  possible. 

7.  Provide  feedback 

7.  Feedback  should  be  chosen  to  fit  the 

nnftv\CA  4l» /«t  f V\/>  niiocfiAn  ic  cawfi t%o  «r\  tits* 

purpose  mat  tne  quesnun  is  serving  in  me 
instructional  process.  It  is  possible  that 
the  type  of  feedback  should  vary  with 
the  learners  performance  and  confidence. 
Give  knowledge  of  correctness  and 
remediation  for  inconrect  answers. 

8.  Assess  performance 

8.  The  purpose  is  to  inform  the  student  of 
progress  toward  mastery.  Feedback 
should  inform  the  student  of  the  adequacy 
of  his  or  her  performance. 

9.  Enhance  retention  and  transfer 

9.  Provide  for  spaced  practice  of  the  newly 
learned  skill  in  an  authentic  situation. 
Immediate  feedback  as  to  correctness 
would  seem  most  appropriate. 
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Event  2:  Informing  the  Learner  of  Objectives.  Questions  may  serve  the  same  purpose 
as  behavioral  objectives  when  they  they  activate  a  process  of  executive  control.  This  is 
supported  by  the  notion  that  schemas  control  perception  and  that  activating  these  schemas 
through  questions  can  affect  selective  perception  (West,  Farmer,  and  Wolff,  1991). 

Sagerman  and  Meyer  (1987)  found  a  learning  effect  that  resulted  from  the  type  of 
adjunct  questions  presented  alter  a  passage  of  text.  Learners  given  verbatim  questions  (i.e., 
questions  in  which  the  learner  was  asked  to  recognize  a  sentence  presented  in  the  text) 
seemed  to  focus  on  verbatim  details  for  subsequent  passages.  Learners  who  received 
conceptual  questions  (i.e.,  questions  that  require  inferring  an  answer  from  information  in  the 
text),  by  contrast,  did  better  on  posttest  conceptual  questions  than  the  verbatim  group.  This 
effect,  which  has  been  referred  to  as  forward  transfer,  or  specific  forward  effect  (Rickards, 
1979),  is  similar  to  the  effect  found  by  researchers  of  performance  objectives.  Faw  and 
Waller's  (1976)  review  of  the  objectives  literature  found  that  (specific)  objectives  focus 
attention  and  improve  performance  on  test  items  related  to  those  objectives  but  depress 
performance  on  incidental  items.  When  combined  with  Rickards1  (1979)  findings,  it  may  be 
that  the  type  of  objective  (detailed  or  general)  might  affect  subsequent  processing  of  the 
passage  more  than  the  content,  since  it  is  unlikely  that  the  learner  will  keep  all  the 
objectives  in  mind  while  reading. 

Event  3:  Stimulate  the  Recall  of  Prerequisite  Learning.  After  the  sensory  registers 
have  recorded  the  information,  selective  perception  takes  place  for  storage  in  short-term 
memory  The  function  of  questions  during  this  stage  would  be  to  aid  the  student  in  recalling 
prerequisites  into  short-term  memory  in  order  to  enable  integration  of  previously  learned 
information  with  new  information  or  skills. 


What  is  learned,  in  most  instructional  situations,  depends  a  great  deal  on  the 
activities  of  the  student.  Rothkopf  (1970)  calls  behaviors  that  give  birth  to  learning 
"mathemagenic"  behaviors.  Similarly,  Kumar  (1971)  points  out  that  which  information 
that  is  transferred  to  short-term  memory  or  long-term  memory  depends  on  the  learner  and  a 
set  of  processes  under  the  learner's  control.  These  processes,  termed  "control  processes11 
(Atkinson  &  Shiffrin,  1968)  include  the  process  of  selective  attention.  The  learner  can 
selectively  extract  information  according  to  his  or  her  perceived  importance  of  the 
information.  This  results  in  a  reduction  of  the  information  present  at  initial  registration, 
since  only  selected  aspects  are  focused  upon  (Kumar,  1971). 

Event  4:  Presenting  the  Stimulus  Material.  Questions  may  be  used  to  enhance  the 
presentation  of  the  instructional  stimulus.  Short-term  memory  (STM)  is  limited  both  in 
duration  and  capacity.  It  is  generally  recognized  that  storage  of  information  lasts  no  more 
than  20  seconds  (usually  less),  and  capacity  insists  of  about  seven  units  of  information  at  a 
time  (Miller,  1956;  Kumar,  1971).  However,  the  duration  of  information  in  STM  can  be 
increased  through  rehearsal,  and  capacity  can  be  increased  by  chunking. 

Questions  at  this  stage  can  serve  to  maintain  and  renew  the  items  stored  in  short- 
term  memory.  They  may  also  aid  learners  in  organizing  information  into  manageable 
chunks  or  propositions.  Feedback  would  serve  to  confirm  the  adequacy  of  rehearsal 
strategies  or  the  propositions  and  provide  reinforcement.  Prompts  or  cues  might  be  used  to 
aid  in  the  process  of  chaining  and  linking  various  sequential  chunks  together  (Anderson, 
1970).  These  hints  could  then  be  faded  as  the  student  becomes  more  proficient  in  the  task. 
Prompts  help  the  learner  respond  correctly  to  a  question  by  focusing  attention  on  relevant 
stimuli  (W.  Wager  &  S.  Wager,  1985).  Two  recognized  types  of  prompts  include  (1)  formal 
prompts,  such  as  spaced  blanks  in  questions  which  denote  the  length  of  the  answer,  and  (2) 
thematic,  contextual,  or  grammatical  cues.  Questions  with  prompts  probably  work  by 
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decreasing  cognitive  load  during  early  stages  of  learning,  and  aid  encoding  by  providing  for 
rehearsal  and  repetition. 

Event  5:  Provide  Learning  Guidance.  Information  which  has  successfully  been 
maintained  in  short-term  memory  must  be  semantically  encoded  for  storage  in  long-term 
memory.  Questions  may  be  used  at  this  stage  to  encourage  learners  to  integrate  new 
knowledge  into  existing  cognitive  structures.  Andre  (1979)  distinguishes  two  distinct 
memory  stores  within  long-term  memory:  (1)  episodic  memory,  which  contains  a  record  of 
events  encountered  and  (2)  semantic  memory,  which  contains  abstracted  or  generalized 
knowledge  such  as  concepts,  principles,  rules,  and  skills  that  are  broader  than  specific 
episodes.  He  suggests  that  this  knowledge  is  represented  as  schema  in  semantic  memory 
and  that  semantic  memory  consists  of  a  network  of  interrelated  schema.  He  also  estimates 
that  higher-level  questions  influence  the  nature  of  the  representation  that  is  formed  when 
learners  acquire  new  information  in  semantic  memory.  Kumar  (1971)  also  suggests  that 
encoding  may  be  the  same  thing  as  schematizing.  The  addition  of  new  knowledge  to 
existing  memory  schemas  is  referred  to  as  accretion,  the  most  common  mode  of  learning 
(Norman,  1982). 

Questions  may  reduce  the  variance  in  processing  information,  thereby  enabling  or 
facilitating  rapid  encoding  (Miller,  1956).  For  example,  in  a  rote  memorization  task  of 
memorizing  a  list  of  words,  the  way  in  which  a  question  is  constructed  can  aid  in  more  rapid 
encoding  of  the  task.  Considei  the  following  list  of  words: 

APPLE  SHIRT 

CHAIR  CARROT 

PANTS  ORANGE 

ONION  PLATE 

CHERRY  SHOES 

TABLE  POTATO 
To  successfully  memorize  the  list  in  30  seconds  may  prove  somewhat  difficult.  However  if  a 
student  is  asked  to  memorize  the  words  as  presented  in  a  new  list  -  one  in  which  the  words 
have  been  put  into  sets  which  allow  for  thern  to  be  remembered  according  to  some  sort  of 
prior  schema  -  the  words  are  much  easier  to  successfully  memorize  in  the  allotted  30 
seconds: 

Categoric 


SHIRT  CARROT 
PANTS  ONION 
SHOES  POTATO 


TABLE  APPLE 

CHAIR  ORANGE 

PLATE  CHERRY 
However,  the  group  given  the  first  list  with  the  question,  "What  four  categories  of 
things  can  the  following  list  be  sorted  into?"  should  be  able  to  nroduce  the  second  list  for 
themselves.  Reigeluth  (1983)  describes  this  as  strategy  activa^on.  If  questions  cause  a 
forward  transfer  effect,  as  described  by  Sagerman  and  Meyer  (1987),  these  types  of  questions 
might  build  a  stronger  use  of  strategies  in  future  learning,  at  least  for  students  who  are  able 
to  apply  the  strategy. 

Feedback  given  in  response  to  questions  which  serve  to  aid  in  semantic  encoding  can 
validate  the  integration  of  new  information  with  pre-existing  knowledge  or  it  may  serve  to 
equalize  question  links  between  existing  structures  and  new  structures  even  further. 
Feedback  also  may  serve  as  learning  guidance  through  the  demonstration  of  processes. 
Elaboration  in  feedback  provides  the  opportunity  for  additional  information  to  be  stored. 
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In  fact,  elaborations  provide  redundancy  in  the  memory  structure  which  can  act  as  a 
safeguard  against  forgetting  and  as  an  aid  in  fast  retrieval  (Reder,  1980). 


Event  6:  Elicit  Performance.  Questions  can  be  used  to  activate  response 
organization.  According  to  Bartlett  (cited  in  Kumar,  1971),  Ha  schema  is  an  active 
organization  of  past  experiences11  (p.  403).  Although  schemas  may  be  different,  they  are 
still  interconnected  by  common  factors  in  an  organized  way.  The  process  of  reconstruction 
activates  a  number  of  relevant  schemas  to  recreate  the  event  (Kumar,  1971).  Questions  can 
act  to  help  in  this  process  of  reconstruction  by  cueing  these  relevant  schemas* 

Post-questions  act  as  confirmation  for  important  mathemagenic  behaviors, 
maintaining  appropriate  responses  on  succeeding  portions  of  text  (Frase,  1970).  Questions 
can  test  for  understanding  or  misunderstanding.  Questions  can  also  act  to  reinforce  learning 
behaviors.  Feedback  acts  to  remediate  misunderstanding  and  to  provide  spaced 
reinforcement. 

Hamaker  (1986)  suggests  that  students  receiving  feedback  to  adjunct  questions  may 
adopt  a  passive  attitude  toward  the  questions  because  they  know  the  correct  answer  will 
eventually  be  presented.  This  would  imply  that  feedback  would  limit  the  amount  of 
information  search  engaged  in  by  a  student  and  lead  to  a  lower  level  of  incidental  learning. 

Event  7:  Provide  Feedback.  When  a  response  is  generated,  the  learner  attempts  to 
confirm  the  correctness  or  appropriateness  of  the  response.  Feedback  serves  to  provide 
reinforcement  and  corrective  information  to  the  student  which  in  turn  works  to  provide  cues 
for  future  performance.  Feedback  can  address  whether  or  not  the  question  was  correctly 
interpreted  and  whether  or  not  the  answer  was  adequate.  Probably  the  most  important  role 
of  feedback  at  this  stage  is  to  correct  misunderstandings. 

Event  8:  Assess  Performance.  Performance  via  a  posttest  or  retention  test  is  probably 
the  most  common  way  of  using  questions.  Assessing  whether  or  not  the  task  can  be  performed 
gi  /es  the  opportunity  for  reinforcement  and/or  validation  through  variations  in  context. 
Feedback  confirms  that  the  skill  has  been  generalized  to  a  range  of  situations. 

Event  9:  Enhance  Retention  and  Transfer.  Knowledge  transfer,  at  the  most  general 
level,  involves  change  in  the  performance  of  a  task  as  a  result  of  the  prior  performance  of  a 
different  task.  The  difference  between  "transfer"  and  f,learning,,  is  that  in  the  case  of 
transfer,  the  two  tasks  are  said  to  be  "different."  In  transfer  paradigms,  the  "different" 
task  is  usually  selected  to  be  novel  to  the  student  (Gick  &  Holyoak,  1987).  The  encoding  of  a 
training  task  will  aid  subsequent  transfer  to  the  extent  that  the  learner  has  acquired  rules 
that  will  be  applicable  to  a  range  of  different  tasks  with  "structural  commonalties." 
According  to  Gick  and  Holyoak  (1987),  "if  the  transfer  task  evokes  similar  goals  and 
processing  mechanisms,  or  has  salient  surface  resemblances  to  the  training  task,  these 
common  components  then  serve  as  the  basis  for  retrieval  of  the  acquired  knowledge  in  the 
transfer  context"  (p.  40). 

One  important  influence  cited  as  affecting  the  encoding  of  the  training  task  is  direct 
feedback  about  performance  levels  (Gick  &  Holyoak,  1987).  Feedback  can  provide  guidance 
and  cues  as  tolvhich  rule  to  apply  in  certain  transfer  problems.  Similarly,  questions  which 
invoke  certain  commonalties  to  be  recognized  and  appropriately  corresponding  rules  to  be 
applied  can  serve  as  a  vehicle  for  transfer. 
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Summary 

Questions  and  Feedback  are  inextricably  related.  Recently  the  process-product 
paradigm  research  on  questions  was  compared  to  a  sociolinguist*  perspective  (Carlson, 
1991).  Carlson  argues  that  the  meaning  of  questions  is  dependent  upon  their  context  in 
discourse,  and  that  the  content  of  questions  cannot  be  ignored.  Some  of  Carlson's  criticisms  of 
research  on  classroom  questions  seem  pertinent  also  for  research  on  feedback.  Feedback  is 
always  related  to  a  response  generated  by  a  question.  In  this  sense,  the  meaning  of 
feedback  is  dependent  upon  its  context  in  the  instruction.  Feedback  effectiveness  may  also 
be  dependent  upon  certain  learner  variables,  such  as  the  learner's  confidence  in  his  or  her 
answer.  But  these  personal  factors  influencing  feedback's  effectiveness  may  vary  according 
to  the  type  of  learning  task  involved  (see  Mory,  1991)  and  may  very  well  have  fluctuating 
influences  within  each  stage  of  information-processing,  as  well. 

In  this  paper,  we  have  attempted  to  show  how  questions  serve  a  variety  of  roles  in 
the  learning  process,  and  so  it  is  reasonable  to  postulate  that  feedback  also  serves  different 
purposes  at  different  stages  in  learning.  Table  1  provides  some  examples  of  how  the  role  of 
the  question  might  determine  the  appropriate  type  of  feedback*  It  seems  unlikely  that  we 
will  find  any  universal  agreement  on  the  ''best11  type  of  feedback.  One  has  to  ask  "Feedback 
for  what?",  and  take  into  consideration  the  type  of  question  the  stage  of  information 
processing,  and  conditions  within  the  learner  to  arrive  at  an  answer. 
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Not  too  long  ago,  I  visited  a  fifth-grade  class  to  observe  a 
student  teacher.    I  had  heard  a  great  deal  about  how  the  children  in 
this  class  made  extensive  use  of  the  computer  for  writing.  According 
to  the  student  teacher,  all  the  children  had  their  own  disks,  and 
using  Appleworks,  they  wrote,  edited,  and  revised  their  stories. 
Eventually,  the  stories  were  "published"  and  became  part  of  the 
class'  personal  library.    The  computers  remained  on  all  day,  and 
students  moved  to  the  computer  area  at  assigned  times  or  whenever 
they  had  a  free  moment. 

As  I  entered  the  classroom,  I  could  see  three  corputers  against 
the  back  wall,  each  in  its  own  narrow  carrel.   Ihe  dividers  were 
clearly  intended  to  ensure  that  the  individual  working  independently 
at  each  computer  wouldn't  be  distracted  by  neighbors.    However,  two 
students  sat  at  each  corputer,  crowded  together,  working  on 
collaborative  writing  activities.   As  I  watched,  I  saw  some 
interesting  "giraffe-like"  behavior:    the  students  at  each  corputer 
would  stretch  their  necks  up  and  around  the  carrel  dividers,  trying 
to  see  what  was  going  on  next  door.    Furthermore,  other  students 
walking  by  (on  their  way  to  the  pencil  sharpener  or  v-he  waste  basket) 
would  stop  and  peer  over  the  shoulders  of  the  students  at  the 
corputer.   They'd  comment  on  what  appeared  on  the  screen,  offer 
suggestions,  correct  spelling,  and  ask  questions.   At  times,  quite  a 
cluster  would  form  in  the  corputer  area. 

Observing  all  this  interaction,  I  recalled— with  a  smile— the 
early  fears  educators  had  voiced  about  microcxxrputers  leading  to 
individual  isolation— fears  that  have  certainly  not  been  borne  out. 
As  Celia  Genishi  (1988)  has  observed,  corputer  use  is  often  "a  social 
event"  (p.  197) .    Clearly,  corputers  can  generate  increased 
coll?l  ^ration  among  children  (Borgh  &  Dickson,  1986;  Dickinson  1986, 
Hawkins  et  al.#  1982;  Wright  &  Samaras,  1986).    One  reason  for  this 
increased  collaboration  is  the  public  nature  of  the  corputer  screen. 
Another  reason  is  the  fact  that  in  most  classrooms,  as  in  the  one  I 
was  visiting,  corputers  are  a  scarce  resource,  so  teachers  often 
assign  children  to  work  in  pairs  or  in  small  groups. 

I've  begun  with  this  anecdote  to  point  out  the  mismatch  that 
sometimes  occurs  between  instructional  goals  and  physical  settings. 
The  teacher  in  this  classroom  actively  encouraged  collaborative 
writing;  she  often  assigned  students  to  work  on  stories  together;  she 
encouraged  group  problem-solving  around  the  computer;  and  she  didn't 
mind  when  students  passing  by  stopped  to  see.  what  was  on  the 
monitor.    Yet  the  physical  setting  was  not  designed  for'  this 
interaction  and  collaboration;  in  fact,  it  was  intended  to  promote 
individual,  independent  task  activity.    The  narrow  carrels  prevented 
two  students  from  sitting  comfortably  in  front  of  each  computer;  the 
computer  area  was  immediately  adjacent  to  students'  desks,  so 
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students  working  on  the  computer  needed  to  be  quiet  in  order  to 
prevent  others  from  becoming  distracted;  and  the  aisle  in  front  of 
the  computer  area  was  narrow  and  not  suitable  for  group  activities 
around  a  monitor. 

When  this  kind  of  mismatch  occurs,  it  underscores  the  need  for 
those  who  design  and  carry  out  instruction — teachers,  curriculum 
developers,  instructional  designers,  trainers—to  attend  to  the 
physical  setting  in  which  instruction  will  occur  and  to  think 
seriously  about  the  way  design  features  can  support  or  hinder 
instructional  goals  (Weinstein,  1981) .   Too  often,  discussions  of 
physical  settings  focus  solely  on  lighting,  acoustics,  the  size  and 
shape  of  the  room,  and  the  location  of  electrical  outlets—the  fixed 
features— and  ignore  environmental  variables  that  can  be  manipulated 
by  users,  like  furniture  arrangement,  clarity  of  pathways  through  the 
space,  amenities,  and  provisions  for  privacy. 

As  my  student  teacher's  story  suggests,  one  physical  variable 
that  needs  to  be  examined  is  seating  arrangement  and  its  impact  on 
interaction  among  students.    We  must  ask  ourselves,  "Given  this 
physical  setting,  how  much  interaction  is  likely  to  occur?"  "How  much 
interaction  do  I  wish  to  foster?"  "What  kind  of  interaction  do  I  want 
to  foster:   conversation,  group  problem  solving,  tutoring?"    "How  do 
I  want  the  interaction  to  flow?"   The  answers  to  these  questions 
should  influence  the  way  we  design  the  space  and,  in  particular,  the 
seating  arrangement  we  choose.   We  know  that  different  seating 
arrangements— horseshoes,  rows,  clusters  of  desks,  tables— can  affect 
interaction  and  task  attention  (Weinstein,  1984) .    Let's  consider 
briefly  about  what  is  known  about  some  of  these  common  seating 
arrangements. 

A  number  of  studies  have  compared  students'  behavior  when  they're 
seated  in  the  traditional  row-and-oolumn  arrangement  with  their 
behavior  when  they're  seated  in  clusters  or  around  tables.  For 
example,  a  1983  study  (Bennett  and  Blundell,  1983)  placed  10-  and 
11-year-old  students  in  a  small  group  seating  arrangement,  then  in 
rows,  then  once  again  in  groups.   One  results  indicated  that  the 
quantity  of  work  completed  inrraaggrf  when  students  were  in  rows, 
although  the  quality  of  work  remained  the  same.    The  teachers  also 
reported  that  there  was  a  noticeable  improvement  in  classroom 
behavior  when  the  students  were  in  rows. 

These  findings  are  consistent  with  earlier  work  (Wheldall, 
Morris,  Vaughan,  &  Ng,  1981;  Axelrod,  Hall,  &  Tarns,  1979) ,  which 
found  that  elementary  students  seated  in  rows  exhibited  greater 
on-task  behavior  than  students  clustered  around  tables.    It  seems 
clear  that  when  the  instructional  goal  is  to  have  students  complete 
individual  tasks,  it  is  unwise  to  place  them  in  clusters,    in  fact,  I 
tell  my  students  that  it  is  inhumane  to  place  students  in  clusters 
and  then  tell  them  that  they  may  not  interact.    Whenever  I  see  this 
situation,  I  am  reminded  of  Phil  Jackson's  (1968)  very  astute 
comments  about  life  in  elementary  classrooms: 

. . .  .students  must  try  to  behave  as  if  they  were  in  solitude,  when 

in  point  of  fact  they  are  not  in  the  early  grades  it  is  not 

uncommon  to  find  students  facing  each  other  around  a  table  while 
at  the  same  time  being  required  not  to  communicate  with  each 
other.    These  young  people,  if  they  are  to  become  successful 
students,  must  learn  how  to  be  alone  in  a  crowd,  (p.  16) 
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On  the  other  hand,  when  the  instructional  goal  requires  students 
to  interact—for  example,  in  coope  \tive  learning  sitmtions  and 
large  group  discussions— tables,  clusters,  squares,  and  circles  are 
definitely  preferable  to  rows.   Peter  Rosenfieid,  Nadina  Lambert,  and 
Allen  Black  (1935)  compared  fifth-  and  sixth-graders'  discussion 
behavior  in  thrae  arrangements — rows,  clusters,  and  circles.  They 
found  circles  were  better  for  discussion  than  clusters,  and  clusters 
were  better  than  rows,  which  produced  more  witlidrawal  and  off -task 
behavior.    One  interesting  observation  was  that  the  cluster 
arrangement  encouraged  students  to  raise  tfca.tr  hands  when  they  had  a 
comment,  whereas  students  seated  in  circles  more  of  ban  made 
spontaneous  "out-of-order"  comments. 

It's  not  difficult  to  see  why  arrangements  like  circles  and 
clusters  would  be  superior  to  rows  for  discussion.  Having 
individuals  sit  face-to-face  promotes  social  interaction  by  providing 
opportunities  for  eye  contact  and  non-verbal  cxitnunication  (e.g. , 
gestures  and  facial  expressions) .   Row  formations,  on  the  other  hand, 
minimize  social  contact  and  thus  help  to  focus  individuals  on  the 
tasks  at  hand. 

In  addition  to  examining  differences  in  interaction  among  seating 
arrangements,  researchers  have  also  looked  at  patterns  of  interaction 
within  arrangements.    For  example,  when  individuals  are  seated  in  a 
circle,  they  are  most  likely  to  make  a  comment  immediately  after  an 
individual  seated  directly  across  the  circle,  and  they  rarely  speak 
to  persons  beside  them  (Steinzor,  1950) .    Similarly,  when  students 
are  seated  in  a  square,  there  is  more  participation  from  people 
directly  opposite  the  instructor  than  from  those  at  the  sides,  and 
students  sitting  adjacent  to  the  instructor  generally  remain  six«nt 
(Sommer,  1967) .    Again,  greater  opportunity  for  eye  contact  and 
non-verbal  communication  appear  to  be  responsible  for  this  pattern  of 
interaction. 

A  comparable  phenomenon  occurs  in  row-and-column  arrangements. 
The  classic  study  in  this  area  was  done  by  Adams  and  Biddle  in  1970, 
These  investigators  found  that  students  who  sat  in  the  front  and 
center  of  the  room  interacted  most  frequently  with  the  teacher 
(assuming  the  teacher  was  in  the  front  and  center  of  the  room) .  The 
effects  were  so  dramatic  that  Adams  and  Biddle  called  this  area  the 
"action  zone."  Apparently,  the  action  zone  phenomenon  is  not  just  a 
matter  of  the  more  interested,  more  eager  students  choosing  seats  in 
the  front.    Although  the  research  is  not  completely  consistent,  there 
is  evidence  that  even  when  seats  are  randomly  assigned,  those  in  the 
front  tend  to  participate  more.    Furthermore,  research  (Schwebel  & 
Cherlin,  1972)  has  indicated  that  when  elementary  students  a.  e  moved 
up  to  the  front,  they  become  more  attentive.    Whether  it  is  increased 
eye  contact  with  the  teacher  or  the  feeling  that  one  is  under  closer 
surveillance,  a  seat  in  the  front-center  of  the  classroom  does  appear 
to  facilitate  participation  (see  Montello,  1988,  for  an  excellent 
review  of  these  studies) . 

In  addition  to  influencing  the  flow  of  communication,  seating 
position  may  affect  an  individual's  perceived  leadership  ability. 
Howells  and  Becker  (1962) ,  for  example,  seated  five-person  groups  at 
a  rectangular  table,  with  two  people  on  one  side  and  three  people  on 
the  other.    Since  the  two  individuals  on  one  side  could  influence 
three  individuals,  and  those  on  the  three-seat  side  could  influence 
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only  two,  the  investigators  hypothesized,  that  members  of  the  two 
person  side  would  emerge  more  frequently  as  leaders.    One  data 
confirmed  this  prediction"-- 14  people  finerged  as  leaders  from  the 
two-seat  side,  compared  with  six  from  the  three-seat  side. 

Vttiat  does  all  this  mean  for  instructional  design?  The  first 
lesson  is  that  the  physical  setting  in  which  instruction  occurs  is 
not  simply  a  neutral  backdrop ,  without  influence  or  importance. 
Indeed,  the  physical  setting  will  affect  learners'  behavior,  whether 
we  intend  it  to  or  not  in  the  instructional  design.    These  effects 
occur  in  two  ways— directly,  by  the  behaviors  the  setting  allows,  and 
indirectly  or  symbolically,  by  the  messages  the  setting  communicates 
about  what  behaviors  are  permitted,  how  important  learning  is,  and 
what  the  roles  of  the  learner  and  teacher  shczld  be  (Proshansky  & 
Wolfe,  1974).    Let  me  give  an  example.   We  know  that  if  individuals 
are  seated  facing  each  other  in  clusters,  they  are  able  to  carry  on  a 
discussion  more  easily  than  if  they  are  seated  apart  from  one  another 
in  rows    Thus,  discussion  is  directly  affected  by  the  arrangement  of 
the  setting.    In  addition,  the  arrangement  may  indirectly  affect 
behavior  by  conveying  the  message  that  the  teacher  values  discussion 
and  collaboration,  that  students  ere  supposed  to  talk.    If  this  is 
indeed  the  message  that  the  teacher  wishes  to  communicate,  all  is 
well  and  good.    If  the  message  is  actually  contrary  to  the  teacher's 
wishes,  we  have  a  situation  where  the  design  of  the  space  contradicts 
the  teacher's  instructional  goals.    In  other  words,  we  have  a 
mismatch  l-oetween  environment  and  intention. 

A  seca-id  lesson  is  that  teachers,  trainers,  and  instructional 
designers  must  consider  the  direct  and  indirect  effects  of  various 
spatial  arrangements  and  determine  which  formations  will  maximize  the 
effectiveness  of  the  designed  instruction.    If  we  do  not 
systematically  analyze  these  effects  and  design  a  setting  to  support 
our  goals,  we  can  easily  become  "victims"  of  the  environment,  for  it 
will  affect  behavior  in  ways  that  we  did  not  intend.    This  process  of 
environmental  analysis  is  somewhat  different  for  those  who  are 
preparing  instructional  materials  and  those  who  are  providing  live 
instruction.    As  Tessmer  and  Harris  (1992)  have  observed,  designers 
are  not  present  to  arrange  seating  when  materials  are  used,  so  they 
must  anticipate  the  environment  in  which  the  activity  will  take  place 
and  provide  some  guidance  on  seating  arrangements  in  instructor  or 
student  manuals. 

Fred  Steele  (1973)  has  coined  the  term  "environmental  competence" 
to  refer  to  an  awareness  of  the  physical  environment  and  its  impact 
and  the  ability  to  use  or  change  that  environment  to  suit  one's 
needs.     Environmentally  competent  teachers  and  designers  do  not 
assume  that  programs,  materials,  and  activities  will  be  equally 
effective  in  any  instructional  environment.    They  do  not  leave  the 
design  of  instructional  settings  to  custodians—who  far  too  often  are 
responsible  for  the  way  our  instructional  settings  are  arranged. 
Instead,  they  ask:    What  will  the  learners  be  doing?   Will  they  be 
reading  or  writing  independently,  will  they  be  engaged  in  cooperative 
learning  activities,  will  they  be  watching  a  videotape,  will  they  be 
collaborating  in  pairs  at  a  microcomputer?   And  then  they  design  a 
physical  arrangement  that  supports  these  activities,  making 
environmental  design  an  integral  part  of  their  instructional  design. 
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A  Critical  Review  of  Elaboration  Theory 

Brent  Wilson  &  Peggy  Cole 
University  of  Colorado  at  Denver 

Elaboration  theory  (ET)  is  a  model  for  sequencing  and  organizing  courses  of 
instruction.  Developed  by  Charles  Reigeluth  and  associates  in  the  late  1970s 
(Reigeluth,  Merrill,  &  Wilson,  1978;  Reigeluth,  Merrill,  Wilson,  &  Spiller,  1979), 
ET  drew  heavily  upon  the  cognitive  research  on  instruction  available  at  the  time, 
in  particular  the  work  of  Bruner,  Ausubel,  and  Norman  (Merrill,  Wilson,  & 
Kelety,  1981).  Since  then,  Reigeluth  has  refined  the  theory  by  offering  detailed 
procedures  for  the  planning  and  design  of  conceptual  (Reigeluth  &  Darwazeh, 
1982),  procedural  (Reigeluth  &  Rodgers,  1980),  and  theoretical  instruction  (see 
Reigeluth  and  Stein,  1983  for  an  overview  and  Reigeluth,  1987  for  a  detailed 
example).  ET  has  been  one  of  the  best-received  theoretical  innovations  in 
instructional  design  (ID)  in  the  last  15  years,  and  is  heavily  referred  to  and  used  by 
practitioners  and  researchers.  At  the  same  time,  research  in  cognitive  psychology 
has  continued  to  shed  light  on  relevant  processes  of  learning  and  instruction.  Just 
as  models  of  learning  change  over  time,  ID  models  also  undergo  regular  changes 
(Merrill,  Kowallis,  &  Wilson,  1981;  Rickards,  1978).  The  purpose  of  this  paper  is  to 
offer  a  critique  of  ET  based  on  recent  cognitive  research,  and  to  offer  suggestions 
for  updating  the  model  to  reflect  new  knowledge.  We  believe  ID  models  should 
undergo  such  revisions  every  few  years  to  stay  current  with  the  growing  knowledge 
base  in  learning,  instruction,  and  other  areas  of  research. 

Elaboration  theory  basics 

ET's  basic  strategies  are  briefly  summarized  below. 

1.  Organizing  structure.  Determine  a  single  organizing  structure  for  the  course 
which  reflects  the  course's  primary  focus.  This  organizing  structure  may  be  one 
of  three  types:  conceptual,  procedural,  or  theoretical.  Reigeluth  (1987)  explains 
that  "In  all  the  work  that  has  been  done  on  sequencing,  elaborations  based  on 
concepts,  principles,  and  procedures  are  the  only  three  we  have  found,  although 
additional  ones  may  be  identified  in  the  future"  (p,  249),  Reigeluth  justifies  the 
use  of  a  single  organizing  structure  by  suggesting  that  "careful  analysis  has 
shown  that  virtually  every  course  holds  one  of  these  three  to  be  more  important 
than  the  other  two"  (Reigeluth,  1987,  p.  248).  The  other  two  types  of  content,  plus 
rote  facts,  "are  only  introduced  when  they  are  highly  relevant  to  the  particular 
organizing  content  ideas  that  are  being  presented  at  each  point  in  the  course* 
(Reigeluth  &  Stein,  1983,  p.  344). 

2.  Simple-to-complex  sequence.  Design  the  coxirse  proceeding  through  the 
identified  structure  in  a  simple  to  complex  fashion,  with  supporting  content 
added  within  lessons.  Begin  with  a  lesson  containing  "a  few  of  the  most 
fundamental  and  representative  ideas  [taught]  at  a  concrete,  application  (or 
skill)  level..."  (Reigeluth,  1987,  p.  248).  This  first  lesson  is  termed  the 
"epitome";  successive  lessons  add  successive  layers  of  complexity  in 
accordance  with  the  categories  of  the  organizing  structure. 

3.  Sequencing  guidelines. 

— For  conceptually  organized  instruction  "present  the  easiest,  most  familiar 

organizing  concepts  first"  (p.  251). 
— For  procedures,  "present  the  steps  in  order  of  their  performance"  (p.  251). 
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— For  theoretically  organized  instruction,  move  from  simple  to  complex. 

•  Place  supporting  content  immediately  after  related  organizing  content. 

•  Adhere  to  learning  prerequisite  relationships  in  the  content1 

•  Present  coordinate  concepts  simultaneously  rather  than  serially. 

•  Teach  the  underlying  principle  before  its  associated  procedure. 

4.  Summarizers  are  content  reviews  (presented  in  rule-example-practice  format), 
at  both  lesson  and  unit  levels. 

5.  Synthesizers  are  presentation  devices — often  in  diagram  form — designed  to 
help  the  learner  integrate  content  elements  into  a  meaningful  whole  and 
assimilate  them  into  prior  knowledge.  They  help  make  content  structure 
explicit  to  the  student;  examples  include  a  concept  hierarchy,  a  procedural 
flowchart  or  decision  table,  or  a  cause-effect  model  with  nodes  and  arrows. 

6.  Analogies  relate  the  content  to  learners'  prior  knowledge.  Effective  analogies 
will  tend  to  bear  strong  resemblance  to  the  content;  weak  analogies  will  contain 
more  differences  than  similarities  with  the  target  content,  Reigeluth  and  Stein 
(1983)  suggest  the  use  of  multiple  analogies,  especially  with  a  highly  divergent 
group  of  learners. 

7.  Cognitive  strat  jy  activators.  A  variety  of  cues — pictures,  diagrams, 
mnemonics,  etc. — can  trigger  cognitive  strategies  needed  for  appropriate 
processing  of  material.  Reigeluth  and  Stein  (1983)  note  that  these  cues  for 
strategy  use  may  be  embedded,  such  as  pictures,  diagrams,  or  mnemonics — 
indirectly  "forcing"  appropriate  processing — or  detached,  such  as  directions  to 
"create  a  mental  image*  of  the  process  you  just  learned"  (p.  362).  Continued  use 
of  these  activators  can  eventually  lead  students  to  understand  when  and  where  to 
apply  various  cognitive  strategies  spontaneously  upon  learning  materials* 

8.  Learner  control.  Reigeluth  and  Stein  (1983)  believe  that  "instruction  generally 
increases  in  effectiveness,  efficiency,  and  appeal  to  the  extent  that  it  permits 
informed  learner  control  by  motivated  learners  (with  a  few  minor  exceptions)" 
(p.  362).  Learners  are  encouraged  to  exercise  control  over  both  content  and 
instructional  strategy.  Clear  labeling  and  separation  of  strategy  components 
facilitates  effective  learner  control  of  those  components.  Regarding  content, 
Reigeluth  and  Stein  (1983)  claim  that  "only  a  simple-to-complex  sequence  can 
allow  the  learner  to  make  an  informed  decision  about  the  selection  of  content" 
(p.  363),  presumably  because  content  choices  will  be  meaningful  ac  any  given 
point. 

KNOWLEDGE  KEPRESENTATION 


Before  turning  to  sequencing  concerns,  we  discuss  the  notion  of  content  structure 
and  its  epistemological  assumptions. 

WHAT  IS  CONTENT  STRUCTURE? 

The  basic  idea  of  content  structure — the  way  content  elements  are  interrelated — 
is  a  long-accepted  notion  in  educational  psychology  (e.g.,  Bmner,  1966).  However, 
the  nature  of  content  structure  is  ambiguous.  Is  content  structure  something 
different  from  people's  cognitive  structures?  Is  there  an  external  body  of 
knowledge  with  its  own  logic  and  form  (Ford  &  Pugno,  1964;  Education  and  the 
structure  of  knowledge,  1964),  or  can  we  only  meaningfully  speak  of  the  structure  of 


1  Wilson  and  Merrill  (1980)  argue  that  both  learning  hierarchy  analysis  and  ET  analysis  result  in 
simple-to-complex  sequencing.  If  this  is  true,  then  searching  for  prerequisite  relationships  in  ET- 
sequenced  skills  is  a  largely  redundant  exercise. 
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individual  understandings?  If  the  distinction  between  external  and  internal 
structures  is  sound,  what  is  the  relationship  between  the  two?  These  questions  are 
substantive  because  instrr^tianal-d^eign  {Jieory  has  been  challenged  in  the  past 
because  of  its  behavicr  \mc  focus  on  eternal  tasks  and  lack  of  attention  to  mental 
structures  and  the  ccgnitivc  mediation  of  learning. 

Certainly,  r>  /ariety  of  fcasfe  analyses  may  be  performed  that  emphasize  different 
aspects  of  'he  task,  many  of  which  do  not  attempt  to  model  cognitive  structure 
(Jon^r  ^n,  Hannun,  &  T'^vner,  :ifi89).  The  same  may  be  said  of  content 
structures.  Content  r*:«y  be  categorized,  analyzed,  and  represented  in  different 
✓ays  toy  different  purposes,  and  need  not  relate  directly  to  internal  cognitive 
repr  sentations.  While  different  positions  may  be  taken,  however,  we  believe  that 
contont/tesk  analysis,  as  a  basic  ID  procedure,  is  most  useful  when  it  models  in 
ex?,r.n&!  form  the  structure  and  process  of  people's  knowledge  and  skills.  Such  a 
model  of  internal  i'-rrns  if  ;*tf>artant  as  a  basis  for  planning  sound  instruction.^ 
TypicaHy,  the  sn«?i  asef/'  k:od  of 1  content  structure11  is  a  model  of  the  way 
knowW^a  is     u^:  t  io  be  structured  in  people's  minds.  Admittedly,  an  external 
uiodtf  nu< r  oe  a  p^m1  f.,pr  .ro:  utmtion  of  people's  knowledge,  but  it  serves  a  useful 
purpc<*  for  planning       designing  instruction. 

HOW  IS  CONTENT  S" Tf^ttTUKSD? 

If  we  accept  the  neJ :.kw  of  extent  structure  as  modeled  cognitive  structure,  the  next 
question  becomes,  vwiat  kinds  of  knowledge  are  there,  and  how  are  they  structured; 
that  is,  how  are  content  elements  interrelated?  Another  way  of  asking  the  question 
is,  how  is  human  knowledge  organized?  As  we  might  imagine,  there  are  as  many 
answers  to  this  question  as  there  are  models  of  human  thought  and  memory, 
ranging  from  simple  chains  of  learned  behaviors  to  complex  networks  to  a  refusal 
to  explicitly  model  human  knowledge  on  the  grounds  that  it  is  inherently  tacit  and 
ineffable. 

Anderson  (1990)  posits  two  basic  kinds  of  knowledge:  declarative  and  procedural 
knowledge.  This  distinction  is  also  made  by  philosophers  (Ryle,  1949)  and  is 
influential  among  educational  psychologists  (E.  Gagne,  1985).  The  distinction  is 
also  popular  with  instructional  designers  (e.g.,  Gagne,  Briggs,  &  Wager,  1988). 
Practicing  teachers  and  designers  make  common  reference  to  "knowledge" 
(declarative  knowledge)  and  "skills"  (procedural  knowledge).  Thus  students  may 
learn  about  computers,  or  they  may  learn  how  to  operate  them.  The  two  forms  of 
knowledge  support  each  other.  Some  theorists  add  image  encoding  as  a  separate 
knowledge  type  (e  g.,  Kosslyn,  1980;  Gagne,  Yekovich,  &  Yekovich,  in  press),  but 
instructional  designers  have  not  emphasized  imagic  knowledge  as  an  indepen- 
dent learning  outcome. 

Seveval  theorists  add  en  integrative  structure  of  some  sort  to  accommodate  these 
knowledge  elements  into  a  whole  pattern,  referred  to  variously  as  schema,  script, 
frame,  or  mental  model  (Norman,  Gentner,  &  Stevens,  1976;  Johnson-Laird, 
1982).  Rumelhart  and  Norman  (1981)  propose  that  all  knowledge  is  procedurally 
represented,  "but  that  the  system  can  sometimes  interrogate  this  knowledge  how  to 
produce  knowledge  that"  that  is,  declarative  knowledge  (p.  343).  Simon  (1980) 
holds  a  similar  view.  Tulving's  (1985)  research  suggests  at  least  three  types  of 


arc  not  naive  to  the  controversy  surrounding  attempts  to  explicitly  model  human  cognitive 
structure  (e.g.,  Dreyfus  &  Dreyfus,  1986;  Churchland,  1984).  It  seems  clear  that  present 
methodologies  are  only  partially  ; successful  at  capturing  human  expertise.  We  are  also  sensitive 
to  connectionist  models  of  cognition  that  emphasize  pattern  matching  over  rule-following 
(Bereiter,  1991;  Bechtel,  1991;  Martindale,  1991).  Nonetheless,  modeling  of  content  structure, 
whatever  its  limitations,  remains  a  valuable  component  of  the  instructional  design  process. 
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memory:  (1)  procedural  memory,  with  a  specialized  subset  of  (2)  semantic 
memory,  with  a  specialized  subset  of  (3)  episodic  memory. 

A  number  of  psychologists  have  added  the  situation  or  context  of  use  as  part  of 
what  gets  learned  (Brown,  Collins,  &  Duguid,  1989).  Rather  than  thinking  of 
expertise  as  the  acquisition  of  a  general  schema,  they  claim  that  learning  and 
expertise  are  always  embedded  in  a  particular  physical,  social,  and  cultural 
context.  Learning  is  a  matter  of  enculturation,  that  is,  of  becoming  part  of  a 
community  which  jointly  constructs  meaning.  Seen  in  this  way,  the  context  of  use 
becomes  part  of  the  "content  structure"  in  need  of  analysis  and  representation  for 
the  design  of  instruction. 

As  mentioned,  there  is  even  some  resistance  to  the  notion  that  human  expertise 
can  be  defined  by  discrete  concepts  and  rules.  Dreyfus  and  Dreyfus  (1986)  argue 
that  real  knowledge  cannot  be  separated  from  the  person,  and  that  reductionist 
attempts  to  model  knowledge  explicitly  are  doomed  to  failure.  In  a  less  radical  but 
equally  compelling  position,  Bereiter  (1991)  challenges  the  idea  common  in 
cognitive  science  that  human  thinking  is  rule-based;  instead,  he  presents  an 
argument  for  viewing  thinking  in  connectionist  terms  as  a  pattern -matching, 
pattern-using  activity. 

At  least  since  Rousseau,  there  has  been  a  strain  of  educational  thought 
opposed  to  the  classical,  rule-based  view  of  learning  and  cognition.  It 
has  often  appealed  to  biological  concepts  of  growth,  emergence,  and 
orgairicism  or  to  social  and  cultural  concepts  and  has  emphasized 
imagination,  spontaneity,  feeling,  and  the  wholistic  character  or 
understanding... .This  strain  of  thought  has  given  rise  to  many 
worthwhile  developments  in  education,  such  as  the... currently  popular 
whole  language  movement.  (Bereiter,  1991,  p.  15) 
The  connectionist  model  thus  rejects  the  partitioning  of  knowledge  into  discrete 
structures  (e.g.,  declarative  and  procedural),  integrating  cognitive,  affective,  and 
psychomotor  aspects  of  performance.  Bereiter  contrasts  this  wholistic  view  of 
learning  with  a  more  rule-based  approach: 

In  contrast  to  the  Rousseauistic  tradition  is  a  family  of  instructional 
theories  in  which  rules,  definitions,  logical  operations,  explicit  proce- 
dures, and  the  like  are  treated  as  central  (Reigeluth,  1983).  From  a 
connectionist  standpoint,  this  family  of  instructional  theories  has 
produced  an  abundance  ot  technology  on  an  illusory  psychological 
foundation.  (Bereiter,  1991,  p.  15) 

Connectionist  theorists  would  clearly  object  to  ETfs  discretely  dividing 
knowledge  into  concepts,  procedures,  and  theories.  Bereiter  concludes  the  article 
by  suggesting  that  the  "situated"  and  "embodied"  cognitive  approaches  could 
provide  a  comprehensive  alternative  that  would  accommodate  elements  of  both 
rule-based  and  connectionist  perspectives  appropriately.  ET  currently  does  not 
provide  detailed  prescriptions  for  making  instructional  sequences  "authentic"  or 
"situated"  in  a  context  similar  to  real-life  problems. 

The  claim  that  not  all  people  solve  problems  by  following  rules  finds  support  in 
research  by  Papert  (1988)  and  Gillian  (1982).  Papert  explains  the  two  ways  bright 
10-  and  11-year-olds  program  computers.  One  way 

fits  the  model  of  "the  logical."  Faced  with  a  problem,  [the  children] 
subdivide  it,  modularize  it,  deal  with  the  parts  one  at  a  time,  put  them 
together  and  make  a  program  that  is  clearly  logically  structured. 
But  other  children  demonstrate  a  different  style—one  in  which  a  pro- 
gram emerges.. .through  something  closer  to  the  way  in  which  a  sculptor 
or  painter  makes  a  work  of  art — a  process  in  which  the  plan  of  what  is  to 
be  made  emerges  and  is  refined  at  the  same  time  as  the  created  object 
takes  form.  (p.  12) 


ERIC 


804 


Elaboration  Theory  Page  5 


Papert  says  the  children  who  use  the  "negotiation al  style  are  performing  at  an 
intellectual  level  that  is  fully  as  excellent  and  of  high  quality  as  the  other  children" 
(p.  12). 

Content  structure  as  organizing  structure 

Recall  that  ET  suggests  using  content  structure  as  an  organizing  and 
sequencing  device,  with  three  main  prescriptions  offered.  Firpt,  courses  and 
lessons  should  be  organized  into  components  according  to  the  content  structure 
being  taught.  This  prescription  is  fairly  broad  and  benign.  The  secor  1 
prescription  is  stronger:  A  course's  organization  should  depend  on  the  primary 
goals  of  instruction:  conceptual,  procedural,  or  theoretical.  If  you  want  learners  to 
have  a  conceptual  overview  of  a  new  subject,  subdivide  and  organize  the  course's 
lessons  according  to  a  taxonomy.  If  your  goals  are  procedural,  begin  with  the 
simplest  version  of  the  procedure  and  progressively  add  more  steps  and  decision 
points;  and  if  your  goals  are  theoretical,  begin  with  the  most  important  principles 
and  add  qualifying  or  extending  principles  in  later  lessons.  The  third 
prescription  is  also  strong:  Course  units  should  ail  reflect  the  primary  organizing 
structure.  That  is,  a  course  with  a  conceptual  structure  as  its  primary  organizing 
structure  should  be  chunked  into  lessons  of  concepts  within  the  original  conceptual 
structure.  A  procedural  course  should  be  chunked  into  increasingly  complex 
versions  of  the  overall  procedure.  The  rationale  for  the  latter  two  prescriptions 
seems  to  be  that  if  the  organizing  structure  entirely  reflects  the  primary  course 
goals  it  will  enhance  meaningful  encoding,  retention  and  retrieval. 

The  first  prescription — that  course  organization  should  basically  reflect  content 
structure — is  consistent  with  text  design  studies  of  access  structure.  As  a  rule, 
students  are  aided  when  text  structure  somehow  reflects  underlying  semantic 
structure  (see,  however,  Mannes  &  Kintsch,  1987,  discussed  below).  The  second 
and  third  prescriptions,  though,  are  much  stronger  versions  of  the  idea.  Again, 
such  constraints  on  designer  judgment  provide  ostensive  gains  in  economy  but 
questionable  payoff.  The  rationale  for  a  single  organizing  structure  seems  to  be 
based  on  assumptions  that  the  development  of  stable  cognitive  structures,  a  goal  of 
ET  (Reigeluth,  1983),  is  best  achieved  by  presenting  content  in  the  framework  of  a 
single,  top-down  organizing  structure.  As  we  will  illustrate  below,  there  are  many 
challenges  to  this  assumption. 

There  seems  to  be  little  evidence  to  draw  on  in  psychology  literature  to  support 
such  a  constrained  approach  to  course  organization.  Posner  and  Strike  (1976; 
Strike  &  Posner,  1976)  reviewed  a  variety  of  methods  for  organizing  courses.  In 
essence,  they  suggest  that  a  course  structure  should  have  a  certain  "face  validity"  to 
the  student;  that  is,  it  should  have  a  logical  and  meaningful  connection  to  students' 
prior  understanding.  The  implication  of  their  review  is  that  courses  need  some 
kind  of  organizing  device  or  logic;  the  precise  kind  of  organization  is  much  less 
important  than  that  it  make  sense  to  the  learner. 

Posner  and  Rudnitsky  (1986)  present  a  somewhat  eclectic  approach  to  course 
design.  Rather  than  three  basic  kinds  of  course  structure,  they  suggest  a  variety  of 
orientations:  inquiry,  application,  problem,  decision,  skill,  or  personal  growth. 
Laurel  (1991)  presents  a  strong  case  for  organizing  computer  interactions  based  on 
a  theatre  metaphor,  involving  the  learner  in  a  stage-like  structure.  This  longer, 
looser  list  seems  to  leave  more  room  for  accommodating  different  kinds  of  course 
and  learning  goals,  as  well  as  prior  knowledge;  moreover,  following  Posner  and 
Rudnitsky,  a  course's  orientation  does  not  constrain  its  sequencing  strategy. 

We  would  argre  for  a  revision  of  ET  that  relaxes  the  connection  between  course 
goals  and  overall  content  structure.  First,  course  goals  can  be  typed  on  a  broader 
basis  than  the  three  goals  listed  by  ET.  Second,  a  variety  of  chunking  strategies 
may  be  useful  for  subdividing  lesson  elements  above  and  beyond  a  single  type  of 
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content  structure.  Designers  need  to  guard  against  rigid  conceptions  of  the  domain 
and  encourage  a  more  dynamic  structure  for  students  to  access  and  learn  at 
various  points  of  instruction. 

DA-STRUCTURED  DOMAINS 

Another  perspective  on  the  structure  of  knowledge  raises  additional  concerns 
about  ET.  Spiro  and  colleagues  (Spiro,  Feltovich,  Coulson,  &  Anderson,  198? ,  Spiro 
&  Jehng,  199G)  became  frustrated  in  their  attempts  to  apply  ID  principles  in 
teaching  complex  and  ill-defined  domains.  This,  according  to  Spiro,  cav  partly  be 
attributed  to  the  fact  that  most  ID  principles  are  based  on  research  using 
introductory  subject  matter.  As  expertise  increases,  however,  the  "content" 
becomes  less  easily  defined,  more  conditional  and  problematic,  and  much  more 
difficult  to  capture  using  traditional  representational  modes.  As  a  consequence 
Spiro  and  colleagues  h  ave  proposed  a  dynamic  view  of  knowledge,  which  they  call 
cognitive  flexibility  tf  eory.  According  to  this  theory,  in  complex  and  ill-defined 
domains,  a  person  ge  lerally  cannot  retri  re  an  intact  schema  from  memory; 
instead,  schemas  combine  or  record-bine  ir  esponSe  to  ihe  requirements  of  each 
particular  situation.  Spiro  and  colleagues  have  developed  an  instructional 
approach  to  facilitate  knowledge  acquisition  in  complex  and  ill-defined  domains, 
criss-crossing  the  domain  with  mini-cases  to  provide  multiple  perspectives  which 
can  later  be  reassembled.  Thcv  ^ecommend  the  use  of  multiple  analogies  and 
cases  to  prevent  the  developm  it  of  oversimplifications  and  misconceptions 
common  among  students  (Spiro,  Feltovich,  Coulson,  &  Anderson,  1989). 

A  key  feature  of  cognitive  flexibility  theory  is  its  view  of  the  subject  matter.  At 
least  for  advanced  knowledge  levels,  content  structure  cannot  simply  be  captured, 
analyzed,  and  used  to  organize  courses.  Advanced  knowledge  is  variable, 
dynamic,  and  ill-defined;  students  in  turn  need  a  variety  of  perspectives  and  expe- 
riences to  appreciate  its  complexity  and  subtlety.  Students  will  tend  to  oversimplify 
and  overgeneralize  when  presented  single  analogies  or  discrete  procedures  and 
rules.  Moreover,  students'  misconceptions  are  fairly  robust  and  resistant  to 
change  (Spiro  et  al.,  1989,  1990).  This  dynamic  view  of  the  subject  matter  seems  at 
odds  with  ET,  which  assumes  that  the  designer  will  organize  instruction  basad  on  a 
well-defined  content  structure. 

ETs  strong  typing  of  knowledge  categories — conceptual,  procedural,  and 
theoretical — is  one  of  its  most  theory-laden  prescriptions.  Constraints  on 
knowledge  representation  might  be  justified  if  there  were  some  kind  of  consensual 
agreement  among  researche  s,  yet  precisely  the  opposite  is  the  case.  According  to 
one  survey  of  educational  literature  in  language  and  cognition,  twenty-five 
distinct  categories  of  knowledge  were  identified  (Alexander,  SchalleH,  &  Hare, 
1991).  Philosophers  and  humanistic  theorists  have  even  more  widely  diverging 
vi*ws  about  the  nature  of  knowledge  and  expertise  (e.g.,  Schon,  1987;  Dreyfus  & 
i>  ayfus,  1986;  Winograd  &  Flores,  1986).  This  is  a  far  cry  from  ETs  three  basic 
knowledge  types!  Indeed,  the  overwhelming  finding  concerning  knowledge 
representation  seems  to  be  that  there  is  no  single  right  way  to  represent  knowledge, 
even  for  a  given  context  or  instructional  purpose.  Even  if  a  course  were  thought  to 
be  primarily  "conceptual"  in  purpose,  a  number  of  diverse  outcomes  are  associated 
with  "conceptual"  learning  (Tessmer,  Wilson,  &  Driscoll,  1990;  Wilson  & 
Tessmer,  1990).  ETs  use  of  conceptual,  procedural,  and  theoretical  structures 
achieves  parsimony  in  its  procedures,  but  at  a  high  cost  to  validity  and  fidelity  to 
current  models  of  learning  and  knowledge. 
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SEQUENCING  ISSUES 

ET  suggests  that  instruction  proceed  from  highly  simplified  representations  to 
gradually  more  complex  content.  While  this  prescription  is  perhaps  amenable  in 
well-structured  domains,  where  expertise  is  clearly  defined,  it  is  problematic  in 
ill-structured  domains.  For  example,  if  the  primary  content  structure  is 
procedural,  ET  would  identify  the  various  paths  through  a  given  procedural 
network,  then  begin  with  the  simplest  version  of  the  procedure;  subsequent 
elaborations  would  merely  add  complexity  to  the  basic  procedure.  But  ET  does  not 
provide  for  the  possibility,  or  even  desirability,  of  two  learners  learning  mutually 
exclusive  procedures.  Recall  that  Papert  (1988)  found  that  student  programmers 
engage  in  mutually  exclusive  styles  (logical  v*.  negotiational).  Also,  Resnick 
(1983)  has  shown  that  math  students  construct  more  sophisticated  procedures  than 
those  taught  in  class.  Thus  an  ID  that  depends  entirely  on  a  single  representation 
of  structure  could  possibly  limit  students'  personal  constructions  of  meaning  from 
the  content. 

A  number  of  theorists  support  the  basic  sequencing  precepts  of  ET.  Bunderson, 
Gibbons,  Olsen,  and  Kearsley  (1981)  suggested  that  instruction  be  geared  around  a 
series  of  work  models,  each  progressing  in  complexity.  Learners  then  "work"  and 
solve  problems  within  a  current  level  of  work  model  until  a  mastery  level  of 
performance  is  reached;  they  are  graduated  to  the  next  level,  which  builds  upon  the 
prior  level.  This  is  similar  in  some  ways  to  White  and  Frederiksen's  (1986) 
approach  that  builds  instruction  around  a  series  of  increasingly  complex 
qualitative  mental  models.  However,  their  approach  begins  with  students1 
intuitive  mental  models,  forcing  students  to  confront  their  misconceptions  and 
develop  increasingly  more  sophisticated  and  correct  menu*1  models.  White  and 
Frederiksen  have  applied  their  simple-to-complex  sequencing  strategy  to  the 
design  of  intelligent  tutoring  systems,  as  well  as  more  traditional  computer-based 
simulations. 

MICROWORLD  DESIGN 

Burton,  Brown,  and  Fischer  (1984),  anticipating  "situated  cognition"  (see 
discussion  above)  used  skiing  as  a  basis  for  studying  the  design  of  learning 
environments  which  they  called  "increasingly  complex  microworlds."  Helping 
novice  performers  "debug"  their  skills  is  a  key  goal  of  microworlds:  "The 
appropriate  microworld  can  transform  'nonconstructive  bugs'  into  'constructive 
bugs/  ones  that  can  be  readily  learned  from"  (Burton,  Brown,  &  Fischer,  1984,  p. 
140). 

Burton  and  colleagues  point  to  three  primary  design  variables  of  skill-based 
microworlds: 

1.  equipment  and  tools  used  in  performing  the  skill; 

2.  the  physical  setting  in  which  the  skill  is  performed; 

3.  the  specifications  for  correctly  performing  the  task. 

The  authors'  notion  of  microworld  design  shares  one  key  design  feature  with 
ET,  that  of  performing  the  simplified  whole  task  whenever  possible: 
Within  each  microworld  that  a  beginning  skier  goes  through,  a 
particular  aspect  of  the  skill  is  focused  on.  But  this  skill  is  not  executed 
in  isolation.  The  student  must  still  do  simplified  versions  of  many  other 
skills  required  to  ski.  Simplifications  of  other  interacting  subskills  let 
the  student  learn  not  only  the  particular  subskill  but  also  how  it  is  used  in 
the  context  of  the  entire  skill,  (p.  150). 
However,  they  differ  from  ET  in  their  emphasis  on  the  means  of  simplification. 
Burton  and  colleagues  encourage  simplifications  of  all  three  design  variables,  but 
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within  real-world  contexts.  They  simplify  equipment  by  recommending  the  use  of 
short  skis  rather  than  long  ones.  They  recommend  simplifying  the  physical 
setting  by  finding  a  downhill  slope  that  feeds  into  an  uphill  slope  so  the  learner  can 
learn  to  glide  without  having  to  learn  at  the  same  time  to  stop.  They  also  simplify 
the  task  itself  by  asking  the  novice  to  practice  gliding  rather  than  traversing.  Thus 
a  variety  of  means  of  task  simplification  are  available  that  go  beyond  wbat  we 
normally  think  of  as  content  structure  (see  also  Wilson,  1985). 

FUNCTIONAL  CONTEXT  TRAINING 

Montague  (1988)  provides  evidence  for  the  effectiveness  of  "functional  context 
training/1  a  spiraling  method  which  begins  with  familiar  objects  about  which 
learners  have  intuitive  knowledge  and  moves  to  progressively  more  complicated 
but  still  familiar  objects.  For  example,  an  introductory  course  for  electronic 
technicians  uses  concrete  and  familiar  objects  for  instruction,  starting  with  a 
flashlight  and  proceeding  to  a  table  lamp,  a  curling  iron,  an  AC  adaptor,  and  a 
soldering  iron.  Instruction  is  situated  in  realistic  settings;  it  integrates  several 
domains  of  knowledge  at  once:  problem  solving,  basic  electricity/electronics 
knowledge,  mental  models  of  devices,  language  processing,  and  mathematics. 
The  sequencing  emphasis  for  the  functional  context  approach  is  to  move  from 
simple-familiar  toward  more  complex-familiar.  This  is  similar  to  ET.  The 
approach  differs  from  ET  in  its  emphasis  on  fidelity  to  job  conditions  and  in- 
context  training.  Also,  rather  than  elaborating  upon  a  single  epitome,  they  use  a 
series  of  concrete  cases  or  analogies,  each  drawing  attention  to  different  aspects  of 
the  subject  area. 

COGNITIVE  APPRENTICESHIPS 

Collins,  Brown,  and  Newman  (1989)  have  described  their  idea  of  the  "cognitive 
apprenticeship."  Like  Montague  (1988),  they  provide  numerous  recommendations 
for  integrating  instruction  with  realistic  performance.  Their  specific 
recommendations  for  sequencing  content,  however,  are  similar  to  those  of  ET  in 
many  ways:  (1)  increasing  complexity,  (2)  increasing  diversity,  and  (3)  global 
before  local  skills.  The  third  recommendation  requires  a  short  explanation. 
Collins  et  al.  suggest  scaffolding  as  a  way  to  support  lower-level  skills  while  the 
student  thinks  about  larger  problems.  "In  algebra,  for  example,  students  may  be 
relieved  of  having  to  carry  out  low-level  computations  in  which  they  lack  skill  to 
concentrate  on  the  higher-order  reasoning  and  strategies  required  to  solve  an 
interesting  problem.. ..The  chief  effect  of  this  sequencing  principle,"  they  explain, 
"is  to  allow  students  to  build  a  conceptual  map,  so  to  speak,  before  attending  to  the 
details  of  the  terrain"  (p.  485).  This  idea  of  supporting  performance  and  helping 
students  develop  clear  mental  models  is  implicit  in  ET  and  certainly  consistent 
with  its  principles  (cf.  Wilson,  1985,  1985-S6). 

Collins  cites  Schoenfeld's  (1985)  math  research  as  a  cognitive  apprenticeship.  He 
has  developed  an  approach  for  teaching  college-level  math  that  employs  a  number 
of  innovative  instructional  strategies.  The  method  focuses  on  guiding  students  to 
use  their  current  knowledge  to  approach  and  solve  novel  problems.  The  instructor 
models  problem-solving  heuristics,  including  the  inevitable  false  starts  and  dead 
ends;  the  process  of  math  problem  solving  is  shown  to  require  creativity  and 
flexibility.  It  is  noteworthy  that  Schoenfeld  sequences  lesson  plans  around 
carefully  selected  cases  that  build  on  each  other  in  a  simple-to-complex  fashion. 
These  cases  are  selected  to  bring  out  certain  features  to  be  learned;  class 
discussions  and  problem  solving  activities  are  flexible  within  the  overall  structure 
of  the  ordered  cases. 
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CASCADED  PROBLEM  SETS 

Schank  and  Jona  (1991)  present  a  sequencing  approach  they  call  cascaded 
problem  sets,  one  of  several  possible  teaching  architectures  they  recommend, 
including  case-based  learning,  incidental  learning,  simulation,  and  directed 
exploration.  Cascaded  problem  sets  rely  on  many  assumptions  similar  to  those  of 
RT;  however,  instead  of  beginning  with  the  simplest  case,  Schank  begins  at  the  end 
and  then  works  backward.  In  essence,  Schank  is  saying,  "We  don't  presume  to 
know  just  what  a  beginning  student  already  knows;  we  prefer  to  give  an  overall 
picture  of  the  final  task  by  starting  at  the  end,  then  work  backwards  to  find  a 
realistic  starting  place  depending  on  the  student's  initial  competency  level." 
The  idea  is  to  build  a  problem  space  whereby  each  problem  relates  to  each 
other  problem  with  respect  to  the  extra  layer  of  complexity  that  it  entails. 
In  other  words,  "if  you  can't  solve  Problem  A,  you  certainly  can't  solve 
Problem  Bw  means  that  B  is  logically  above  A,  Between  A  and  B  would  be 
some  information  that  B  entails  that  A  does  not.  Below  A  would  be  some- 
thing simpler  than  A  that  perhaps  does  not  entail  the  knowledge  common 
to  A  and  B.  As  students  have  trouble  with  one  problem,  they  move  down 
the  cascade  of  problems  by  learning  about  the  issues  that  one  would  need 
to  know  to  solve  the  problem  they  were  having  trouble  with.  (Schank  & 
Jona,  1991,  pp.  20-21,  italics  added.) 

Another  difference  with  ET  is  that  Schank  considers  task  components  as  part  of 
the  cascaded  problem  set.  "A  problem  must  be  broken  down  into  its  constituent 
parts.  Each  constituent  would  itself  be  a  problem,  and  it  too  would  have  constituent 
parts... .For  example,  at  the  bottom  of  a  cascade  of  algebra  problems  would  be  basic 
arithmetic"  (Schank  &  Jona,  1991,  p.  20).  This  parts  analysis  seems  more 
reminiscent  of  Gagne's  learning  hierarchies  than  ETs  meaningful  spiraling.  In 
either  case,  the  idea  of  cascaded  problem  sets  is  clearly  derivative  of  well- 
established  ID  principles,  including  work  on  computer-adaptive  testing,  even 
though  the  authors  do  not  cite  previous  work  on  the  probiem. 

MIDDLE-OUT  SEQUENCING 

ETs  conceptual  structures  are  sequenced  from  the  top  down,  that  is,  from  the  most 
general  conceptual  category  down  to  the  most  detailed  sub-category  in  a  taxonomy. 
We  have  criticized  this  approach  elsewhere  (Wilson  &  Cole,  in  press  a)  for  its 
failure  to  accommodate  learners'  prior  knowledge.  Our  basic  point  is,  why  teach 
the  concept  Vertebrate'  before  'cow'  to  a  small  child,  just  because  it  happens  to  be 
higher  in  a  conceptual  hierarchy?  Lakoff  (1987)  makes  a  similar  point;  he  reviews 
a  large  body  of  literature  suggesting  that  in  normal  settings,  people  tend  to  classify 
and  think  about  objects  at  &  "middle  level,"  not  too  general  and  not  too  detailed. 
Rosch  (Rosch,  Mervis,  Gray,  Johnson,  &  Boyes-Braem,  1976)  gives  the  term  'basic 
categories'  to  this  level  of  natural  perception.  For  example,  people  tend  to  think  in 
terms  of  dogs  (basic  level)  rather  than  animals  (superordinate  level)  or  retrievers 
(subordinate  level).  Similarly,  'chair'  is  more  psychologically  basic  than 
'furnituie'  or  'rocker.'  Rosch  suggests  that  most  of  our  knowledge  about  the  world  is 
organized  at  this  level;  most  attributes  pertaining  to  category  membership  are 
stored  at  this  middle  level.  She  also  suggests  that  this  basic  level  of  category  is: 

•  The  highest  level     which  category  members  have  similarly  perceived 
overall  shapes. 

•  The  highest  level  at  which  a  single  mental  image  can  reflect  the  entire 
category. 

•  The  highest  level  at  which  a  person  uses  similar  motor  actions  for  interacting 
with  category  members. 

•  The  level  at  which  subjects  are  fastest  at  identifying  category  membnrs. 

•  The  level  with  the  most  commonly  used  labels  for  category  members. 
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•  The  first  level  named  and  understood  by  children.  (Lakoff,  1987,  p.  46) 
If  people  organize  their  knowledge  primarily  around  these  basic-level  categories, 
then  it  seems  unreasonable  to  insist  on  proceeding  in  a  strict  general-to-detailed 
order  down  a  taxonomy.  A  more  defensible  strategy  would  start  from  learners' 
prior  schemas,  then  proceed  both  up  and  down  the  taxonomy  into  new  territory,  as 
increasingly  difficult  but  authentic  tasks  require.  Tessmer  (1991)  terms  this  a 
"middle-out"  sequencing  strategy,  where  instruction  begins  at  a  middle  level  of 
generality,  gradually  adding  both  superordinate  and  subordinate  detail.  This 
alternative  sequencing  strategy  represents  a  significant  revision  of  ETs  design 
prescriptions. 

SEQUENCING  FOR  CONCEPTUAL  CHANGE 

A  line  of  cognitive  research  investigates  instructional  interventions  that  try  to 
link  up  with  learners'  preconceptions  and  schemas  about  the  world  (e.g.,  Siegler, 
1991);  this  body  of  research  is  sometimes  referred  to  as  conceptual  change 
literature.  Some  researchers  (e.g.,  Case  &  Bereiter,  1984;  Resnick,  1983;  White  & 
Frederiksen,  1986)  have  directly  challenged  models  that  order  instruction  based  on 
subject  matter  logic  and  neglect  learners'  existing  conceptions.  Preexisting 
conceptions  may  be  a  help  or  a  hindrance  to  new  learning;  misconceptions  and 
"buggy*  procedures  can  often  interfere  with  the  assimilation  of  new  skills  and 
knowledge.  Case  (1978)  developed  an  instructional  model  in  which  the  teacher 
directly  confronts  learners'  misconceptions;  after  learners  see  clearly  the  inade- 
quacy of  their  existing  conceptions,  they  become  ready  to  acquire  new  models,  and 
will  tend  to  integrate  the  new  knowledge  more  directly  into  their  current 
structures.  The  general  strategy  for  conceptual  change  is  : 

1.  Learners  must  become  dissatisfied  with  their  existing  conceptions. 

2.  Learners  must  achieve  at  least  a  minimal  understanding  of  an  alternate 
way  of  conceptualizing  the  issue. 

3.  The  alternative  view  must  appear  plpusible. 

4.  Learners  must  see  how  the  new  conceptualization  is  useful  for  understanding 
a  variety  of  situations,  (based  on  Bransford  &  Vye,  1989) 

Whereas  an  ET-style  lesson  might  proceed  smoothly  through  a  content  structure, 
conceptual-change  lessons  proceed  in  fits  and  starts,  working  from  student 
misconceptions,  failing,  trying  again,  beginning  false  starts,  :  creating  from 
dead  ends,  each  time  elaborating  upon  students'  schematic  understandings  (cf. 
also  Schoenfeld,  1985).  The  "elaboration"  is  not  on  an  external  content  structure, 
but  rather  on  an  internal  representation. 

Thus  the  conceptual  change  literature  emphasizes  the  dynamic  nature  of 
learning.  Two  observations  are  particularly  relevant  here.  First,  learners' 
understandings  result  from  the  interplay  of  their  prior/existing  knowledge  and  the 
current  instructional  situation  (e.g.,  Mayer,  1980).  Second,  we  cannot  anticipate 
students'  emergent  mental  models;  they  may  be  riddled  with  "bugs"  or  more 
sophisticated  than  a  course's  terminal  objective  (e.g.,  Resnick,  1983).  ID  must 
accommodate  this  dynamic,  often  chaotic  situation  (e.g.,  Jonassen,  1990;  Winn, 
1990). 

A  number  of  conceptual-change  researchers  draw  heavily  on  Vygotsky's 
notion  of  a  "zone  of  proximal  development,"  wherein  children  can  perform  tasks 
with  the  help  of  adult  "scaffolding"  and  assistance  (Wertsch,  1985).  Vygotsky's 
approach  would  sequence  tasks  so  as  to  keep  learners  engaged  in  tasks  that  stretch 
them  to  go  beyond  their  present  level  of  expertise,  but  which  can  be  performed  with 
social  support  and  appropriate  tools  and  information  resources. 

In  line  with  Vygotsky's  zone  of  proximal  development,  Newman,  Griffin,  and 
Cole  (1989)  think  of  tasks  as  something  accomplished  by  groups  of  people.  They 
contrast  their  approach  with  traditional  ID.  Following  traditional  ID, 
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First,  the  tasks  are  ordered  from  simple  or  easy  to  complex  or  difficult. 
Second,  early  tasks  make  use  of  skills  that  are  components  of  later 
tasks,  Third,  the  learner  typically  masters  each  task  before  moving  onto 
the  next.  This  conception  has  little  to  say  about  teacher-child  interaction 
since  its  premise  is  that  tasks  can  be  sufficiently  broken  down  into 
component  parts  that  any  single  step  in  the  sequence  can  be  achieved  with 
a  minimum  of  instruction.  Teacherless  computerized  classrooms 
running  "skill  and  driir  programs  are  coherent  with  this  conception  of 
change.  (Newman,  Griffin,  &  Cole,  1989,  p.  153) 
The  authors  contrast  this  tas'c-analysis  approach  with  a  more  teacher-based 
approach  where  simplicity  is  achieved  by  the  social  negotiation  between  teacher 
and  learner: 

The  teacher  and  child  start  out  doing  the  task  together.  At  first,  the 
teacher  is  doing  most  of  the  task  and  the  child  is  playing  some  minor 
role.  Gradually,  the  child  is  able  to  do  more  and  more  until  finally  he 
can  do  the  task  on  his  own.  The  teacher's  actions  in  these  supportive 
interactions  have  often  been  called  "scaffolding,1'.. .suggesting  a  tempo- 
rary support  that  is  removed  when  no  longer  necessary.. ..There  is  a 
sequence  involved. ..but  it  is  a  sequence  of  different  divisions  of  labor. 
The  task — in  the  sense  of  the  whole  task  as  negotiated  between  the  teacher 
and  child — remains  the  same.  (p.  153) 

This  sequencing  method  is  less  analytic  and  formal  than  ET,  yet  the  end  product 
of  the  social  interaction  is  usually  a  simple-to-complex  sequencing  of  "content." 
This  approach,  like  the  skiing  example  above,  offers  a  more  flexible  view  of  content 
and  ways  of  simplifying  instruction. 

INTERNAL  REFLECTION-IN-ACTION  PROCESSES 

Schon's  (1983,  1987)  reflective  practitioner  model  sees  professionals — doctors, 
lawyers,  architects,  teachers,  etc. — as  embodying  personal  theories  of  practice. 
These  personal  theories  are  much  more  important  than  academic  theories  or 
representations  of  expertise.  When  professionals  (or  aspiring  professionals) 
encounter  a  problem  in  everyday  work,  much  of  their  response  is  routinized,  but 
there  is  nonetheless  an  element  of  on-the-spot  reflection  and  experimentation. 
Schon  describes  a  typical  learning  sequence  of  reflection-in-action: 

•  First  we  bring  routinizod  responses  to  situations.  These  responses  are  based 
on  tacit  knowledge  and  are  "spontaneously  delivered  without  conscious 
deliberation."  The  routines  work  as  long  as  the  situation  fits  within  the 
normal  range  of  familiar  problems. 

•  At  some  point,  the  routine  response  results  in  a  surprise — an  unexpected 
outcome,  positive  or  negative,  that  draws  our  attention. 

•  The  surprise  leads  to  reflection-in-action.  We  tacitly  ask  ourselves  "What's 
going  on  here?"  and  "What  was  I  thinking  that  led  up  to  this?" 

•  Through  immediate  reflection,  we  re-examine  assumptions  or  recast  the 
problem  in  another  way.  We  may  quickly  evaluate  two  or  three  new  ways  to 
frame  the  problem. 

•  We  engage  in  an  "on-the-spot  experiment*  We  try  out  a  new  perspective  or 
understanding  of  the  situation,  and  carefully  note  its  effects.  The  cycle  of 
routine  performance — surprise — interpretation — experiment  is  repeated  as 
needed,  (adapted  from  Schon,  1987,  p.  28) 

Schon  rejects  the  validity  of  traditional  academic  formulations  of  expertise.  The 
traditional  discipline— its  theories,  concepts,  models,  etc. — simply  does  not  capture 
the  personal  expertise  needed  to  reason  and  evaluate  in  a  professional  capacity.  By 
extension,  we  could  argue  that  instructional  designers  simply  cannot  capture, 
represent,  and  teach  the  "content  structure"  really  needed  for  expertise.  That 
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expertise  lies  embedded  within  the  expert  practitioner,  and  can  only  be  acquired 
through  extended  opportunities  of  practice  in  authentic  settings,  with  appropriate 
coaching,  mentoring,  and  other  guidance  with  feedback.  The  guidance  is  less  in 
the  form  of  general  principles  and  rules,  and  more  in  the  form  of  contextualized 
reasoning  based  on  the  specifics  of  a  case.  Because  the  domain  is  ill-structured,  the 
practitioner  cannot  always  routinely  activate  an  intact  schema;  instead  the 
practitioner  must  assemble  a  new  schema,  combining  and  recombining 
knowledge  from  many  cases  in  memory. 

Viewed  differently,  the  reflective-practitioner  model  provides  a  convincing 
portrait  of  the  way  that  general  models  of  ID  relate  to  the  evr  yday  practice  of  ID 
professionals.  There  is  a  growing  indication  that  instructional  designers  do  not 
apply  formal  models  in  a  lock-step  fashion.  Indeed,  ID  models  often  fail  to  capture 
expert  designers9  knowledge  and  skill.  This  common  problem  between  theory  and 
practice  is  aggravated  when  the  "prescriptive"  ID  models  are  represented  in  a 
highly  technical  and  rigidly  proceduralized  fashion.  We  return  to  this  point  in  the 
recommendation  section  below. 

Putnum  (1991)  reports  some  interesting  research  in  which  he  observed  how 
consultants  grew  in  their  expertise  in  using  Schon's  reflective  practitioner  model 
with  teachers.  Putnum  notes: 

Many  of  us  who  seek  to  engage  people  in  significant  learning 
experiences  disparage  formulas,  rules,  or  recipes  for  action  as 
superficial. ...Novices  are  likely  to  misuse  rules  and  recipes;  they  have 
not  developed  the  know-how  to  use  them  correctly. 
Yet  well-intentioned  learners  do  search  for  rules  and  recipes,  espe- 
cially early  in  a  learning  process.  As  one  participant  said  after  a  work- 
shop on  promoting  organizational  learning,  "If  you  could  only  give  us  a 
list  of  the  eight  things  to  say,  that  would  be  really  helpful  in  getting 
started."  This  person  was  not  naive;  he  understood  that  a  handful  of 
recipes  was  not  a  substitute  for  genuine  mastery.  The  difficulty  is  that  a 
new  theory  of  practice  cannot  be  acquired  whole.  Yet  if  it  is  acquired 
piecemeal,  the  pieces  are  likely  to  be  used  in  ways  that  violate  the  whole, 
(p.  145) 

Schon's  model  includes  several  techniques  that  Putnum  calls  "recipes."  In  a 
general  sense,  "a  recipe  is  a  formula,  a  set  of  instructions,  for  designing  action" 
(Putnum,  1991,  p.  147).  In  his  research,  Putnum  studied  a  particular  kind  of  recipe, 
a  question  fragment  ("What  prevents  you  from...?")  used  as  a  technique  in 
consulting  situations.  Putnum  reports  that  consultants  seemed  to  progress  through 
stages  of  competence  in  their  use  of  these  fragments: 

1.  Novices  use  recipes  as  "one-liners"  or  invariant  procedures.  "Lacking 
experience  in  the  theory  of  practice  from  which  the  recipe  was  drawn,  novices 
may  get  themselves  in  trouble  they  cannot  get  themselves  out  of.  Nevertheless, 
they  may  feel  a  sense  of  success  at  having  done  what  they  are  'supposed  to  do/ 
what  they  believe  an  expert  might  have  done.  At  the  same  t:  ne  they  may  feel 
some  discomfort  or  chagrin  at  imitating  or  'being  a  parrot"'  (Putnum,  1991,  p. 
160). 

2.  The  novice  gradually  shifts  orientation  from  the  recipe  itself  to  broader 
strategies  and  concepts.  Still,  "learners  may  remain  caught  in  a  kind  of  tunnel 
vision,  concentrating  intently  on  the  mechanics  of  implementing  the  new 
strategy.  It  is  therefore  difficult  to  respond  flexibly  to  the  [dynamic  feedback]  of 
the  situation"  (p.  160). 

3.  Eventually,  learners  become  able  to  "respond  to  surprising  data  by  reframing 
the  situation,  stepping  out  of  their  original  perspective  to  take  account  of 
another."  Learners'  attitudes  about  recipe-following  also  shift:  "Rather  than 
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feeling  successful  simply  by  using  a  recipe,  they  may  consider  whether  that 
usage  was  pro  forma  or  genuine"  (p.  161). 

Putnum  points  to  three  positive  functions  of  recipes  that  serve  to  counterbalance 
the  negative  effects  of  their  misuse  by  novices.  First,  they  serve  to  elicit  useful  data 
in  practice  situations.  That  data  can  then  serve  as  feedback  to  learners  in 
improving  their  practice.  Second,  recipes  tend  to  have  memorable  phrases  which 
can  serve  as  hooks  or  mnemonics  to  aid  performance.  Thus  a  recipe  may  act  as  a 
retrieval  cue  to  activate  an  appropriate  schema  for  a  given  situation.  Third,  the 
concrete,  memorable  nature  of  recipes  also  can  aid  problem  encoding  and 
reflection.  Students  often  organize  their  reflective  thoughts  around  preexisting 
recipes. 

The  reflective  practitioner  model  is  relevant  tc  ET  because  it  highlights  how 
learners  themselves  construct  and  organise  meaning  in  a  basically  simple-to- 
complex  way.  Novices  take  what  they  can  from  the  content.  Then,  given  authentic 
performance  opportunities  and  appropriate  coaching  and  reflection,  surface-level 
imitation  proceeds  to  a  kind  of  problem  solving  based  on  deeper  understanding  of 
the  situation.  As  Resnick  (1983)  stated,  "Effective  instruction  must  aim  to  place 
learners  in  situations  where  the  constructions  that  they  naturally  and  inevitably 
make  as  they  make  sense  of  their  worlds  are  correct  as  well  as  sensible  ones"  (p. 
31).  Rather  than  being  "presented"  the  content  structure,  learners  construct  the 
content  for  themselves  through  reflective  processes.  Thus  a  simple-to-complex 
progression  may  occur,  even  if  the  external  "content"  remains  the  same:  the  same 
recipe  comes  to  mean  something  entirely  different  to  an  experienced  practitioner. 

MAKING  CONTENT  STRUCTURE  EXPLICIT 

ET  suggests  that  content  structure  be  made  explicit  to  students  through  various 
synthesizers  and  organizers.  This  approach  is  in  line  with  most  research  on  text 
design  (e.g.,  Jonassen,  1982,  1985).  However,  findings  of  Mannes  and  Kintsch 
(1987)  and  McDonald  (1988),  challenge  the  conventional  wisdom  about  organizing 
devices  and  synthesizers.  They  found  that  presenting  students  an  outline 
consistent  with  the  text  structure  fostered  memory-level  encoding  but  impeded  far 
transfer  of  the  material  to  problem-solving  tasks.  This  finding  may  be  related  to 
Smith  and  Wedman's  (1988)  comparison  between  instruction  sequenced  according 
to  ET  prescriptions  and  Gagne-style  learning  hierarchies.  They  found  that 
students  made  more  meaningful  elaboration  upon  the  learning  hierarchy 
sequenced  material,  even  though  the  ET  materials  were  more  meaningfully 
ordered  and  presented.  It  soems  possible  that  highly  structured  and  clearly  ordered 
materials  may  allow  superficial  encoding  precisely  because  of  their  easy  access 
structure. 

These  possible  negative  effects  of  explicit  teaching  of  structure  may  be  related  to 
the  reported  negative  effects  of  constant  knowledge-of-results  feedback  for  motor 
learning  tasks  (Salmoni,  Schmidt,  &  Walter,  1984):  When  the  student  has  to  do 
less  work  to  make  sense  of  things,  less  learning  may  occur.  Salmoni  et  al. 
distinguish  between  immediate  performance  in  instruction  and  delayed 
performance  as  a  measure  of  learning.  They  suggest  that  certain  instructional 
strategies  may  result  in  a  performance  decrement  during  practice,  but  that  on  a 
retention  task,  the  strategies  may  result  in  learning  gains.  Thus  the  possibility 
may  be  entertained  that  ordering  instruction  in  a  too  facile  way  could  result  in 
minimal  dissonance  and  could  ironically  result  in  shallow  processing  of  material 
by  students  (Wilson  &  Cole,  1991  b).  Salomon  and  Sieber  (1970)  provide  some 
evidence  for  this  interpretation.  They  hypothesized  f,(a)  that  a  randomly  spliced 
film  arouses  states  of  uncertainty  which  in  turn  lead  the  learner  to  extract 
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information  concerning  possible  interpretations  of  [the]  film,  and  (b  when  the  film 
is  well  organized,  it  provides  a  structure  for  remembering  details"  (Salomon,  1974, 
p.  394).  This  is  an  area  that  is  threatening  to  key  concepts  in  ID;  there  is  an  obvious 
need,  however,  for  further  research  before  strong  claims  can  be  supported. 

Constructivism  has  recently  gained  prominence  as  a  philosophy  of  cognitivism; 
ID  theorists  currently  are  exploring  implications  for  practice  (e.g.,  Educational 
Technology,  May  and  September  1991).  A  constructivistic  approach  to  instruction 
is  reflected  in  Harel  and  Papert's  (1990)  teaching  of  fractions  through  Logo. 
Students'  learning  of  fractions  was  reinforced  by  their  designing  computer  lessons 
which  taught  something  about  fractions.  Through  the  process  of  designing  the 
lessons,  students  came  to  understand  the  procedures  and  concepts  of  fractions  at  a 
deeper  level  than  a  control  group.  Their  knowledge  of  Logo  programming, 
fractions,  and  problem-solving  skill   significantly  exceeded  those  of  both  a  Logo- 
programming  group  and  a  control  group.  (The  design  group,  however,  took  propor- 
tionately longer  on  task.)  The  fractions  study  reflects  a  growing  emphasis  among 
cognitive  researchers  in  design  and  composition  activities  as  a  method  of  learning 
new  knowledge  (Harel,  1991),  An  analogy  might  be  the  student  journalist  who 
learns  a  lot  about  both  street  crime  and  writing  by  doing  a  series  of  stories  on  the 
subject.  Currently,  ET  does  not  directly  address  the  issue  of  building  instruction 
around  design  activities. 

Constructivistic/connectionistic  approaches  also  tend  to  stress  coaching 
environments  (Burton  &  Brown,  1979;  Rossett,  1991)  and  inquiry-learning 
strategies  (McDaniel  &  Schlager,  1990;  Collins  &  Steven,  1983).  The  conditions  of 
appropriate  use  of  a  variety  of  alternative  sequencing  strategies  go  beyond  ET 
prescriptions  and  need  to  be  more  clearly  articulated  for  instructional  designers  to 
be  able  to  make  appropriate  design  decisions. 


As  research  in  cognitive  psychology  continues  to  shed  light  on  the  process  of 
learning,  we  are  forced  to  reexamine  the  assumptions  and  prescriptions  of  various 
theories  of  ID,  including  ET.  We  have  explained  what  we  believe  are  some  of  the 
stronger  challenges  to  ET.  Following  are  some  of  the  clear  implications  for 
change,  which  we  believe  may  require  a  radical  restructuring  of  ET,  particularly 
if  it  is  to  serve  the  needs  of  instruction  in  complex  and  ill-defined  domains: 
1.  Deproceduralize  the  theory.  In  its  current  form,  ET  is  less  a  theory  and  more  of  a 
design  procedure.  Explicit  steps  are  provided  for  designing  and  sequencing 
instruction.  This  procedural  approach  has  two  problems  associated  with  it:  (a) 
the  procedural  prescriptions  often  go  far  beyond  our  knowledge  base  about 
learning  and  instructional  processes,  and  are  often  at  odds  with  that  knowledge; 
and  (b)  instructional  designers  tend  to  follow  models  in  a  principle  based, 
heuristic  manner  in  spite  of  detailed  procedural  specifications  (Taylor,  1991; 
Wedman  &  Tessmer,  1990;  Nelson  &  Orey,  1991;  Schon,  1983).  ET  should  be 
reformulated  into  a  set  of  guiding  principles  referenced  more  clearly  to 
learning  processes.  A  principle-based  formulation  will  allow  practicing 
designers  to  adapt  the  concepts  to  a  greater  variety  of  instructional  situations. 
Thn  key  principles  of  a  revised  version  of  ET  seem  to  be: 
#  All  subject  matters  have  an  underlying  content  structure,  i.e.,  how  people 
relate  constructs  together  meaningfully,  This  structure,  however,  is 
personally  idiosyncratic  and  dynamic,  particularly  in  complex  domains. 
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•  The  modeled  structure  of  the  content  should  be  taken  into  account  in  organiz- 
ing and  sequencing  courses  and  lessons.  Overall  sequence  should  generally 
proceed  from  simple  to  complex,  allowing  for  the  great  variety  of  ways  to 
move  toward  increasing  complexity, 

•  The  content  structure  should  ultimately  be  made  explicit  to  the  student  The 
specific  mechanisms  (e.g.,  direct  instruction  versus  inquiry  methods) 
should  be  determined  by  the  instructional  situation  (learner  characteristics, 
goals,  setting,  need  for  efficiency,  etc.). 

2.  Remove  unnecessary  design  constraints.  A  number  of  ET  prescriptions 
constrain  designer  options  without  a  demonstrable  return  in  the  form  of 
instructional  effectiveness.  Examples  include 

•  the  use  of  three  primary  structures  (conceptual,  procedural,  and  theoretical), 

•  tying  together  the  primary  ct  irse  goal  and  primary  organizing  structure, 
and 

•  using  a  single  structure  as  a  basis  for  organizing  the  entire  course. 
These  prescriptions  make  ETs  application  more  standardised  and 

parsimonious,  but  they  also  preclude  a  number  of  alternative  organization 
schemes  that  follow  the  "spirit"  of  ET  but  not  the  "letter,"  Again,  a  principle- 
based  formulation  of  ET  could  more  easily  accommodate  variant  schemes, 

3.  Base  organization  and  sequencing  decisions  on  learners"  understandings  as 
well  as  the  logic  of  the  subject  matter.  An  assumption  implicit  in  ET  is  that  the 
simplest,  most  general  concepts  in  a  subject  are  also  the  closest  to  learners'  prior 
understanding.  We  have  shown  this  assumption  to  be  unfounded.  An 
p]ternative  emphasis  would  be  to  add  these  heuristics: 

•  Move  from  familiar  to  less  familiar  content. 

•  Use  content  with  high  interest  and  perceived  relevance  (Hidi,  1990). 

•  Create  and  then  take  advantage  of  the  "teaching  moment"  (Bransford  &  Vye, 
1989)  when  learners  are  receptive  and  prepared  for  new  ways  of  looking  at 
things.  Induce  cognitive  conflict,  e.g.,  by  presenting  an  anomaly  (Perkins, 
1991),  then  help  learners  accommodate  new  information  into  their  existing 
schemas. 

•  Respond  to  emergent  mental  models,  encouraging  learners  to  confront  their 
misconceptions. 

•  Wherever  possible,  ground  instruction  in  an  authentic  performance  setting. 
Make  heavy  use  of  immediate,  concrete  situations,  tools,  problems,  and  forms 
of  feedback. 

4.  Assume  a  more  constructivist  stance  toward  "concent  structure"  and  sequencing 
strategy.  An  objectivistic  view  of  content  is  that  it  is  "out  there";  a  constructivist 
view  claims  that  content  is  in  people's  minds,  generated  through  a  process  of 
social  negotiation,  and  can  only  be  loosely  modeled  externally  (Cunningham, 
1991).  We  can  only  hope  to  approximate  an  accurate  representation  of  true 
expertise;  much  of  an  expert's  knowledge  is  tacit  and  ineffable,  resistant  to 
reduction  and  analysis.  On  this  view,  a  desig-ner's  understanding  of  the 
content  can  guide  selection  of  learning  experiences,  but  cannot  directly  control 
learning  outcomes  in  a  direct,  engineered  way. 

Author  notes 

Please  send  requests  for  reprints  to  Brent  Wilson,  University  of  Colorado  at 
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Abstract 

Teachers  average  one  decision  every  two  minutes  while  they  are  teaching.  The  kinds  of 
decisions  they  make  affect  their  students.  One  hundred  twenty-two  undergraduate  and 
graduate  education  majors  were  tested  to  explore  what  decisions  they  would  make  when 
confronted  with  problem  statements  referring  to  the  planning  and  delivery  of  instruction. 
Results  indicated  that  experienced  teachers  were  better  able  to  identify  problem 
situations  and  choose  the  appropriate  missing  component  than  novice  teachers. 
Instructional  design  principles  can  help  teachers  recognize  problems  and  make 
appropriate  changes  in  their  instruction. 
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Introduction 

It  has  been  proposed  thnt  if  instructional  design  (ID)  principles  were  incorporated 
into  the  classrooms  by  teachers,  "students  would  become  motivated,  instruction  would 
become  dear  and  logical,  student  achievement  would  increase,  teachers  would  be  freed 
from  the  drudgery  of  routine  tasks,  and  classroom  activities  would  become  more  varied" 
(Kerr,  1989  p.  7).  This  would  be  accomplished  because  ID  presents  alternatives  to 
traditional  teacher-based  instruction  and  research  has  shown  that  utilizing  the  process 
creates  an  effective  means  of  planning  classroom  instruction.  However,  at  present  most 
classroom  teachers  have  not  received  formal  training  in  the  procedures  commonly  used  by 
instructional  designers.  If  ID  is  to  help  novice  teachers  move  from  a  stage  of  being 
basically  reactive  (Kerr,  1981)  to  a  point  where  they  become  more  skilled  in  handling  the 
"infonnation  processing  load  and  decision  making  during  interactive  teaching"  (Walter, 
1984  p.  8),  these  basic  principles  must  be  taught  in  current,  formal,  teacher  training 
programs. 

Davis  and  Silvernail  (1983)  concluded  from  their  national  study  of  recognized 
curriculum  experts  that  "curriculum  design  and  instructional  design  skills  are  viewed  as 
essential  parts  of  teacher  preparation  programs  but  that  not  all  of  these  skills  are 
actually  represented  in  the  curricula  of  such  programs."  Reiser  (1989)  goes  on  to  predict 
by  the  year  2001  "it  is  likely  that  teachers  in  the  public  schools  will  pay  more  attention  to 
design  principles,  but  it  is  unlikely  that  the  school  systems  will  employ  many 
instructional  designers." 

One  model  for  planning  effective  instruction  propounded  by  Dick  and  Reiser  (1989) 
consists  of  the  xbllowing  seven  components  in  order:  motivation,  objectives,  prerequisites, 
information  ard  examples,  practice  and  feedback,  testing,  enrichment  and  remediation. 
Research  suggests  that  this  sequence  of  events  results  in  the  most  efficient  and  effective 
approach  to  learning  (Dick  &  Reiser,  1989).  It  was  hypothesized  that  if  teachers  -  both 
novice  and  experienced « were  given  a  short  description  of  these  seven  components  they 
would  be  better  able  to  make  instructional  decisions. 

To  determine  the  kind  of  instructional  decisions  teachers  make  when  faced  with 
everyday  situations,  an  instrument  containing  the  seven  components  of  an  instructional 
plan  with  a  short  description  and  an  additional  instrument  containing  the  seven 
components  without  a  description  was  used.  This  was  accompanied  by  14  typical 
classroom  scenarios  for  the  teachers  to  read  and  make  a  decision  regarding  which 
component  may  be  missing. 

Hie  Current  Study 

The  purpose  of  this  study  was  to  examine  the  ability  of  novice  and  experienced 
teachers  -  not  formally  trained  in  ID  principles  -  to  identify  the  components  of  effective 
instructional  plans.  This  is  in  response  to  a  growing  trend  in  the  public  schools  that 
requires  teachers  to  use,  or  at  least  be  aware  of,  some  of  the  basic  principles  of 
instructional  design. 

Method 

Subjects 

The  subjects  were  122  undergraduate  and  graduate  education  majors  from  a  large 
southwestern  university.  Sixty- two  subjects  were  identified  as  novices  (two  or  less  years 
of  teaching  experience,  x  =  .23  years)  arid  sixty  were  identified  as  experienced  (three  or 
more  years  of  teadiing,  x  -  7.45  years).  Among  the  novices,  13  Were  early  education 
majors,  31  elementary,  15  secondary,  and  tliree  special  education  Among  the 
experienced  teachers,  five  held  degrees  in  early  education,  17  in  elementary,  30  in 
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secondary,  and  eight  in  special  education.  All  of  the  subjects  were  enrolled  in  a  basic 
teacher  preparation  class  which  must  be  taken  at  the  beginning  of  their  education  block 
In  addition,  all  of  the  subjects  were  volunteers  and  were  not  given  any  extra  points  for 
participating. 

Materials 

An  instrument  was  developed  containing  14  typical  comments  that  might  be  heard 
from  students  during  or  after  studying  a  particrlar  topic.  Each  of  the  seven  components  in 
an  effective  instructional  plan  (Dick  &  Reiser,  1989)  was  inferred  as  missing  two  different 
times. 

Half  of  the  instruments  contained  a  short  description  of  each  of  the  seven  components 
-  motivation,  objectives,  prerequisites,  information  and  examples,  practice  and  feedback, 
testing,  enrichment  and  remediation.  The  remaining  half  of  the  instalments  gave  a 
listing  of  the  seven  components  but  did  not  contain  any  descriptions.  A  larger  number  of 
the  test  materials  than  subjects  was  provided. 

Procedures 

Materials  were  distributed  so  that  approximately  one  half  of  the  subjects  would 
receive  an  instrument  with  a  short  description  of  the  components,  plus  the  14  comments 
and  the  remainder  would  receive  instruments  with  a  only  list  of  the  components,  plus  the 
14  comments. 

All  of  the  subjects  participated  in  groups  within  their  respective  classroom  setting. 
Subjects  were  randomly  assigned  to  the  two  conditions  -  descriptions/no  descriptions  - 
based  on  their  seating  order.  Thirty*  two  of  the  novices  and  28  of  the  experienced  teachers 
received  test  materials  with  definitions.  Thirty  of  the  novices  and  32  of  the  experienced 
teachers  received  test  materials  without  definitions. 

Subjects  were  instructed  to  read  each  comment  and  choose  which  one  of  the  seven 
components  may  be  missing  from  the  instructional  plan  A  typical  comment  reads  "I 
studied  and  studied,  but  I  never  really  understood  what  it  was  I  was  supposed  to  be 
learning".  The  missing  component  in  this  example  shows  that  the  teacher  had  not  made 
the  objective  clear  to  the  student.  Subjects  were  mformti  that  there  was  no  time  limit 
and  were  encouraged  to  take  as  long  as  they  felt  they  needed. 

Criterion  Measure 

An  instrument  was  developed  based  on  the  examples  given  in  R.    ing  Effective 
Instruction  (Dick  &  Reiser,  1989,  p.  82-83).  Fourteen  comments  which  might  be  heard 
from  a  student  in  a  typical  classroom  setting  were  given/constructod  to  reflect  the  absence 
of  one  of  the  seven  components  described  as  essential  to  an  instructional  plan  There 
were  two  comments  per  component.  The  instrument  was  given  to  graduate  level 
instructional  design  students  to  test  the  reliability  between  the  two  seta  of  comments  - 
those  created  by  the  author  and  those  used  by  Dick  and  Reiser.  The  r  suiting  measure  of 
equivalence  and  stability  was  .99.  The  number  of  correct  responses  U4  possible)  to  the 
comments  was  used  as  the  criterion  measure  for  data  analysis. 

Design  and  Data  Analysis 

The  design  was  a  2(experienoe)  x  2(definition)  factorial.  An  analysis  of  variance 
(ANOVA)  was  used  to  analyze  the  data  from  the  experiment 
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Results 

The  mean  scores  by  treatment  are  shown  in  Table  1  (the  overall  mean  performance 
score  was  8.21  out  of  14  possible).  Experienced  teacher*  had  a  mean  ncore  of  9  out  of  a 
possible  14.  Novice  teachers  had  a  mean  score  of  7.5.  The  average  score  for  subject*!  in 
the  two  description  groups  was  8.35  compared  to  R  06  for  subjects  in  the  no  description 
groups.  Items  missed  by  more  than  50%  of  novices  per  tained  to  objectives,  testing,  and 
enrichmentvtemediation.  The  only  items  missed  by  more  than  50%  of  experienced 
teachers  pertained  to  enrichment/remediation. 


Insert  Table  1  about  here 


A  preliminary  analysis  was  conducted  using  analysis  ot  variance  (ANOVA) 
procedures  to  determine  the  effects  of  experience  and  component  descriptions  on 
performance.  Alpha  was  set  at  .05  for  all  statistical  t  -ate. 


Insert  Table  2  ah  >ut  here 


Analysis  of  the  posttest  data  indicated  that  giving  a  short  description  of  each  of  tiu; 
seven  components  did  not  significantly  affect  performance  f\3, 118)  -  541  p  .05. 

Results  did  indicate  a  significant  difference  between  experienced  and  novice  teacher 
for  identifying  the  components  missing  from  an  effective  hiatrurtional  plan  F(3, 118)  == 
10.67,  p_<  .001. 

Discission 

Ihe  purpose  of  this  paper  is  to  descrifo :  the  i^esulte  of  a  atudy  oonau<rted  to 
detennine  the  ability  of  teadiers  (novice  and  tixperienoed)  to  identify  basic  components  oi 
an  instructional  plan.  There  are  numerous  instructional  design  models  1o  aasist  in  ih 
task  of  designing  classroom  instruction  aid  making  effective  decisions  concerning  its 
implementation.  These  models  have  long  been  available  to  instructional  designers,  yet 
teachers  are  expected  to  learn  ID  principles  through  a  process  of  trial  and  error  in  the 
daily  classirom  setting.  This  study  pnrvidf  is  the  basis  of  an  argument  fbrincoipomting 
instructional  design  principles  into  preservice  and  inservioe  teacher  preparation 
programs. 

The  results  of  this  study  suggest  that  experienced  teacht  rs  -  without  the  benefits  of 
formal  instructional  design  training  -  develop  an  ability  to  identify  ID  problems  in  an 
instructional  plan.  This  was  evidenced  by  the  significantly  higher  performance  of  the 
experienced  teachers  in  identifying  instructional  problems,  a  basic  step  in  inntructional 
design.  An  obvious  explanation  might  be  that  teachers  do  mom  than  merely  deliver  <  ^ 
implement  instruction  (Snelbecker,  1987).  In  fact,  Clark  and  Peterson  (19ti(i)  detenruuod 
that  teachers  average  at  leakt  one  instructional  derision  every  two  minutes.  With  such  a 
significant  amount  of  daily  practice  in  making  instructional  derisions,  it  appears  that 
many  teachers  acquire  bits  and  pieces  of  the  information  pertaining  to  effective 
instruction  that  could  and  should  be  presented  to  them  in  the  teacher  education 
programs. 

The  variable  of  providing  short  descriptions  of  the  components  appeared  to 
very  little  difference  to  both  the  novice  and  experienced  tearhrm  Tlii«  was  ps\>bably  iht<« 
to  the  simplistic  natur  e  of  the  component  names  themselves.  (Jenerally,  teathow  -  noviu1 
and  experienced  alike  -  havo  Ixnm  ox\yumd  to  such  M>els  as  motivation,  ohjnrUv;! 
prerequisites,  etc.  in  their  teacher  edwation  wogntmn,  parten>  iiwiioraiMieioiices,  woiii 
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scttingn  or  otlierwiao,  ar  !  therefore  tlie  component  names  thernttelves  and  their 
meanings  were  easily  understood  by  the  subjects. 

Future  research  in  this  <<rea  might  focus  on  the  ootnponenta  tJiat  were  understood 
the  leaat  -  objectives,  testing,  oiirichmenttemecUation  -  and  therefore  make  up  for  the 
majority  of  the  errors  hi  tho  results  of  this  study.  Also,  wliat  is  the  best  method  for 
incorporating  instnictional  design  into  teachn  piupai  kUon  programs  aixi  how  much 
oveiiap  ahoidri  theiv  fa*  in  teacher  preparation  andinstrii  tiona.  tofcigh  prugiums? 
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Table  1 

Mean  Performance  Scores  by  Teacher  Experience  and  Presence  of  Description 
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Presence  of  Description 


Level  of  Teacher  Experience 


Description 


No  Description 


Total 


Novice 

M 
SD 

Experienced 

M 
SD 


Total 


M 

SD 


7.59 
2.08 


9.11 
1.69 


8.35 
2.04 


7.33 
2.36 


8.78 
1.68 


8.06 
2.15 


746 

2.2 


8.95 
1.68 


8.21 
2.09 


Notn.  n=62  for  novices;  n=60  for  experienced.  All  scores  are  out  of  a  maximum  of  14 
possible. 
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Table  2 

ANOVA  Summary  Table  for  Teacher  Perfomianoe 


Source  of  Variation 

df 

MS 

F 

E 

Main  Effects 

44.871 

2 

22.436 

5.516 

.005 

Definition? 

2.201 

1 

2.201 

.541 

.463 

Experience 

43.404 

1 

43.404 

10.672 

.001 

Definition  X  Experience 

1.886 

1 

1.886 

.464 

.497 

Explained 

46.757 

3 

15.586 

3.832 

.012 

Within  Cells 

479.907 

118 

4.067 

Total 

526.664 

121 

4.353 
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Answers  to  Test  Comments 

1.  E  -  Practice  and  Feedb.  ^ 

2.  A -Motivation 

3.  G-  Enrichment 

4.  D  -  Information  and  Examples 

5.  F  or  E  -  Testing  or  Practioe  and  Feedback 

6.  C  -  Prerequisites 

7.  B  -  Objectives 

8.  E  -  Practice  and  Feedback 

9.  A -Motivation 

10.  B-  Objectives 

11.  F- Testing 

12.  C  -  Prerequisites 

13.  D  -  Information  and  Examples 

14.  G- Enrichment  and  Remediation 
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